-- Logs begin at Tue 2022-06-21 17:14:15 UTC, end at Tue 2022-06-21 18:57:08 UTC. -- Jun 21 18:10:24.648398 dv-cinder-844601-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 21 18:10:24.662979 dv-cinder-844601-4-iscsi devstack@keystone.service[80387]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 21 18:10:24.665811 dv-cinder-844601-4-iscsi devstack@keystone.service[80387]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Jun 21 18:10:24 2022] *** Jun 21 18:10:24.665811 dv-cinder-844601-4-iscsi devstack@keystone.service[80387]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 21 18:10:24.665811 dv-cinder-844601-4-iscsi devstack@keystone.service[80387]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 21 18:10:24.665811 dv-cinder-844601-4-iscsi devstack@keystone.service[80387]: nodename: dv-cinder-844601-4-iscsi Jun 21 18:10:24.665811 dv-cinder-844601-4-iscsi devstack@keystone.service[80387]: machine: x86_64 Jun 21 18:10:24.665811 dv-cinder-844601-4-iscsi devstack@keystone.service[80387]: clock source: unix Jun 21 18:10:24.665811 dv-cinder-844601-4-iscsi devstack@keystone.service[80387]: pcre jit disabled Jun 21 18:10:24.665811 dv-cinder-844601-4-iscsi devstack@keystone.service[80387]: detected number of CPU cores: 4 Jun 21 18:10:24.665811 dv-cinder-844601-4-iscsi devstack@keystone.service[80387]: current working directory: / Jun 21 18:10:24.665811 dv-cinder-844601-4-iscsi devstack@keystone.service[80387]: detected binary path: /usr/bin/uwsgi-core Jun 21 18:10:24.665811 dv-cinder-844601-4-iscsi devstack@keystone.service[80387]: your processes number limit is 47680 Jun 21 18:10:24.665811 dv-cinder-844601-4-iscsi devstack@keystone.service[80387]: your memory page size is 4096 bytes Jun 21 18:10:24.665811 dv-cinder-844601-4-iscsi devstack@keystone.service[80387]: detected max file descriptor number: 2048 Jun 21 18:10:24.667383 dv-cinder-844601-4-iscsi devstack@keystone.service[80387]: lock engine: pthread robust mutexes Jun 21 18:10:24.667383 dv-cinder-844601-4-iscsi devstack@keystone.service[80387]: thunder lock: enabled Jun 21 18:10:24.667383 dv-cinder-844601-4-iscsi devstack@keystone.service[80387]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 21 18:10:24.667383 dv-cinder-844601-4-iscsi devstack@keystone.service[80387]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 21 18:10:24.730291 dv-cinder-844601-4-iscsi devstack@keystone.service[80387]: Python main interpreter initialized at 0x55774f17c9c0 Jun 21 18:10:24.730291 dv-cinder-844601-4-iscsi devstack@keystone.service[80387]: python threads support enabled Jun 21 18:10:24.730291 dv-cinder-844601-4-iscsi devstack@keystone.service[80387]: your server socket listen backlog is limited to 100 connections Jun 21 18:10:24.730291 dv-cinder-844601-4-iscsi devstack@keystone.service[80387]: your mercy for graceful operations on workers is 90 seconds Jun 21 18:10:24.730840 dv-cinder-844601-4-iscsi devstack@keystone.service[80387]: mapped 403077 bytes (393 KB) for 2 cores Jun 21 18:10:24.730957 dv-cinder-844601-4-iscsi devstack@keystone.service[80387]: *** Operational MODE: preforking *** Jun 21 18:10:24.731725 dv-cinder-844601-4-iscsi devstack@keystone.service[80387]: *** uWSGI is running in multiple interpreter mode *** Jun 21 18:10:24.731725 dv-cinder-844601-4-iscsi devstack@keystone.service[80387]: spawned uWSGI master process (pid: 80387) Jun 21 18:10:24.732043 dv-cinder-844601-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 21 18:10:24.733985 dv-cinder-844601-4-iscsi devstack@keystone.service[80387]: spawned uWSGI worker 1 (pid: 80399, cores: 1) Jun 21 18:10:24.733985 dv-cinder-844601-4-iscsi devstack@keystone.service[80387]: spawned uWSGI worker 2 (pid: 80400, cores: 1) Jun 21 18:10:24.733985 dv-cinder-844601-4-iscsi devstack@keystone.service[80387]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 21 18:10:32.847636 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 21 18:10:32.847636 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: from cryptography.utils import int_from_bytes Jun 21 18:10:32.848786 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 21 18:10:32.848786 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: from cryptography.utils import int_from_bytes Jun 21 18:10:32.932530 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 21 18:10:32.932819 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 21 18:10:32.933169 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] command line args: [] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 21 18:10:32.933587 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 21 18:10:32.933815 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ================================================================================ {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 21 18:10:32.934247 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] admin_token = **** {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:32.934669 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] config_dir = [] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:32.935006 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:32.935339 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] config_source = [] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:32.935667 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] debug = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:32.936032 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:32.936650 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:32.936984 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:32.937469 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:32.937968 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:32.938396 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] list_limit = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:32.938735 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] log_config_append = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:32.939054 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:32.939390 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] log_dir = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:32.939725 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] log_file = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:32.940185 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:32.940652 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:32.941105 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:32.941576 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:32.941925 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:32.942234 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:32.942699 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:32.943162 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:32.943606 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:32.944037 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:32.944476 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:32.944878 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:32.945323 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:32.945822 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:32.946335 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:32.947273 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.87/identity {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:32.947669 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] publish_errors = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:32.948103 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:32.948539 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:32.948875 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:32.949346 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:32.949827 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:32.950277 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] standard_threads = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:32.950705 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:32.951165 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:32.951498 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:32.951815 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] use_journal = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:32.952157 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] use_json = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:32.952486 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] use_stderr = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:32.952915 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] use_syslog = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:32.954035 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:32.954035 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.954497 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.954674 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.955051 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.955503 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.955942 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.956396 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.956824 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.957275 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.957766 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.958228 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.958662 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.959125 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.959563 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.960049 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.960514 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.960962 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.961450 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.962032 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.962402 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.962570 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.962817 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.963285 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.963731 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.964171 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.964580 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.964981 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.965380 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.965744 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.966014 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.966312 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.966567 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.966808 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.967104 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.967376 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.967598 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.967899 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.968088 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.968318 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.968559 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.968795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.969030 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.969263 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.969497 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.969771 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.970028 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.970260 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.970513 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.970746 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.970985 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.971224 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.971474 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.971704 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.971944 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.972351 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.972534 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.972770 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.973023 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.973416 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.973714 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.973979 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.974290 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.974527 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.974801 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.975085 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.975266 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.975490 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.975729 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.976005 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.976324 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.976581 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.976878 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.977080 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.980185 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.980185 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.980185 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.980185 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.980185 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.980185 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.981248 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.981248 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.981248 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.981783 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.982048 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.982529 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.982939 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.983338 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.983761 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.984219 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.984633 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.985042 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.985464 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.985907 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.986284 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.986536 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.986791 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.987039 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.987294 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.987542 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.987868 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.988166 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.988434 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.988768 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.989187 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.989600 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] auth.external = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.990126 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.991069 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.991324 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.991570 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] auth.password = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.991813 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] auth.token = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.992066 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.992366 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.992560 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.992797 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.993050 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.993308 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.993543 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.993997 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] credential.caching = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.994494 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.994925 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.995374 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.995803 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.996230 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.996641 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.997090 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.997526 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.998023 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.998467 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.998951 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.999716 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:32.999859 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.000404 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.000752 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.001177 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] federation.caching = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.001594 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.002241 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.002748 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.002955 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.003356 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.003837 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.004265 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.004668 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.005072 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.005492 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.006010 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.006442 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity.caching = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.006859 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.007291 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.007711 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.008135 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.008569 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.008997 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.009446 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.009996 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.010455 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.010899 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.011334 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.011763 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.012205 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.012656 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.013081 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.013534 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.014048 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.014559 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.015011 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.015435 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.015837 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.016263 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.016667 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.017091 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.017528 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.017997 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.018443 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.018852 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.019290 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.019742 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.020130 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.020573 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.021010 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.021429 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.021901 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.022395 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.022820 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.023239 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.023675 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.024111 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.024532 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.024976 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.025415 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.025909 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.026899 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.026899 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.027309 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.027880 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.028409 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.028711 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.029151 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.029461 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.029788 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.030047 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.030314 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.030580 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.030842 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.031113 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.031403 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.031684 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.031970 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.032513 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.032983 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.033305 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.033761 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.034209 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.034675 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.035112 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.035512 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.035968 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.036389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.036792 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.037210 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.037626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.038085 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.038517 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.038931 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.039386 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.039791 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.040217 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.040655 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.041033 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.041722 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.042083 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.042537 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.042937 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.043352 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.043767 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] resource.caching = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.044207 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.044641 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.045064 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.045510 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.046011 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.046449 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.046897 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.047337 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.047780 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.048212 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] role.caching = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.048667 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] role.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.049087 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.049575 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.050046 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.050513 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.050982 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.051450 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.051923 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.052376 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.052851 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.053281 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.053786 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.054242 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.054709 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.055149 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.055889 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.056317 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.056878 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.057238 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.057698 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.058139 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.058580 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.059056 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.059517 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.059968 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.060409 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.060841 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.061224 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.061717 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.062165 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.062442 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.062770 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.063041 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.063370 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.063739 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] token.caching = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.064105 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.064375 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.064721 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.065074 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.065336 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.065583 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.066015 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.066324 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.066527 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.066774 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.067028 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.067268 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.067520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.067781 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.068014 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.068261 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.068566 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.068835 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.connection = **** {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.069094 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.069346 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.069609 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.069914 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.070146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.070507 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.070803 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.071111 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.071450 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.071705 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.072039 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.072367 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.072703 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.073157 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.073518 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.073923 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.074173 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.074472 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.074663 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 21 18:10:33.109900 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:33.110157 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:33.110329 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:33.120778 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:33.160073 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:33.160307 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:33.327449 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:33.349936 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 21 18:10:33.349936 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: from cryptography.utils import int_from_bytes Jun 21 18:10:33.350906 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 21 18:10:33.350906 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: from cryptography.utils import int_from_bytes Jun 21 18:10:33.390489 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:33.390826 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:33.440730 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 21 18:10:33.441061 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 21 18:10:33.441400 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] command line args: [] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 21 18:10:33.441962 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 21 18:10:33.442280 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ================================================================================ {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 21 18:10:33.442751 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] admin_token = **** {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:33.443212 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] config_dir = [] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:33.443552 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:33.443896 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] config_source = [] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:33.444234 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] debug = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:33.444605 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:33.445037 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:33.445377 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:33.445901 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:33.446350 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:33.446780 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] list_limit = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:33.447108 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] log_config_append = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:33.447477 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:33.447802 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] log_dir = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:33.448151 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] log_file = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:33.448630 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:33.449124 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:33.449611 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:33.450098 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:33.450424 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:33.450759 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:33.451206 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:33.451677 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:33.452111 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:33.452549 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:33.453007 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:33.453435 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:33.453926 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:33.454373 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:33.454874 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:33.455721 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.87/identity {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:33.456056 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] publish_errors = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:33.456484 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:33.456929 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:33.457256 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:33.457772 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:33.458229 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:33.458688 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] standard_threads = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:33.459096 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:33.459538 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:33.460029 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:33.460297 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] use_journal = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:33.460534 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] use_json = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:33.460819 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] use_stderr = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:33.461156 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] use_syslog = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:33.461612 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:10:33.462700 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.463220 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.463715 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.464284 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.464759 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.465218 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.465717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.466156 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.466618 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.467083 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.467573 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.468027 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.468501 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.468698 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:33.468894 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:33.469079 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.469260 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:33.469260 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:33.469683 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.469870 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:33.469870 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:33.470208 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.470403 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.470814 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.471294 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.471768 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.472224 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.472734 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.473251 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.473767 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.474237 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.474672 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.475105 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.475470 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.475840 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.476218 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.476605 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.476986 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.477341 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.477744 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.478125 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.478503 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.478976 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.479394 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.479844 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.480280 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.480712 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.481142 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.481569 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.482049 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.482502 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.483000 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.483386 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.483835 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.484251 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.484646 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.485024 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.485411 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.485795 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.486175 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.486553 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.486930 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.487345 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.487788 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.488218 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.488705 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.489147 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.489603 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.490025 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.490745 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.491151 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.491539 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.491992 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.492422 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.492853 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.493217 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.493607 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.494035 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.494492 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.494876 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.495355 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.495708 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.496113 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.496549 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.496997 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.497437 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.497890 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:33.498099 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.498290 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:33.498463 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.498627 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:33.498627 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:33.498930 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:33.499087 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.499265 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:33.499265 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:33.499585 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.499745 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.500043 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.500480 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.500913 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.501393 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.501901 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.502346 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.502808 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.503264 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.503717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.504196 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.504603 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.505038 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.505488 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.505969 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.506436 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.506879 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.507355 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.507865 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.508309 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.508732 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] auth.external = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.509173 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.509773 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.510512 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.510684 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] auth.password = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.511122 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] auth.token = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.511583 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.512041 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.512238 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:33.512519 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.512957 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.513436 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.513937 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.514389 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.514837 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] credential.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.515305 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.515786 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.516265 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.516692 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.517141 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.517589 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.518112 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.518597 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.519054 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.519529 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.520055 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.520518 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.521029 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.521517 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.522045 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.522504 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] federation.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.522976 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.523431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.523886 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.524321 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.524783 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.525228 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.525786 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.526243 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.526728 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.526873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:33.527044 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:33.527181 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.527295 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:33.527488 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:33.527611 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.527796 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:33.527935 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:33.528060 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.528251 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:33.528550 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.528988 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.529430 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.529900 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.530363 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.530801 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.531247 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.531727 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.532222 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.532687 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.533110 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.533538 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.534050 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.534531 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.534985 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.535478 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.535962 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.536470 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.536947 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.537413 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.537904 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.538381 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.538839 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.539289 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.539748 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.540202 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.540675 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.541162 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.541683 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.542164 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.542636 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.543105 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.543601 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.544046 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.544497 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.544942 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.545400 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.545925 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.546393 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.546853 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.547258 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.547403 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.547529 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 21 18:10:33.547783 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.548072 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.548267 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.548403 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 21 18:10:33.548691 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.548820 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.549006 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.549161 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.549295 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.549491 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.549618 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.550140 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.550613 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.551053 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.551488 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.551904 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.552384 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.552882 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.553318 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.553823 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.554290 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.554775 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.555252 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.555709 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.556164 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.556641 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.557149 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.557586 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.558300 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.558778 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.559243 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.559678 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.560137 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.560569 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.561055 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.561528 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.562042 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.562535 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.562993 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.563459 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.563939 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.564385 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.564869 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.565440 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.565951 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.566419 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.566891 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.567362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.567677 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.567847 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.570194 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.570385 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.570522 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.570522 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.570762 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.570762 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.570762 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] resource.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.570762 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.570762 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.570762 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.571406 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.571406 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.571641 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.571990 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.573137 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.573609 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.574085 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] role.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.574550 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] role.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.574980 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.575459 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.575894 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.576311 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.576717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.577114 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.577620 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.578048 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.578505 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.578937 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.579417 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.579898 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.580730 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.580879 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.581034 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:10:33.581622 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.582078 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.582542 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.583222 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.583479 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.583991 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.584459 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.584648 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:10:33.584946 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.585077 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.585228 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.585419 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.585720 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.585890 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.586021 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.586172 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.586302 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.586447 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.586580 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.586710 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.586922 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.587057 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.587316 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.587467 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.587588 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.587739 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.588213 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.588643 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.589016 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.589368 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.589822 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] token.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.590204 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.590584 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.591001 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.591455 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.591916 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.592373 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.593006 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.593419 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.593825 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.594279 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.594904 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.595117 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.595571 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.596037 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.596488 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.596967 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.597520 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.598043 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.connection = **** {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.598515 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.598975 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.599427 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.599881 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.600327 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.600770 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.601195 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.601597 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.602045 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.602480 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.602938 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.603395 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.603932 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.604361 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.604801 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.605230 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.605740 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.606165 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:10:33.606496 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 21 18:10:33.628374 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.631247 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:10:33.631709 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.642041 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:10:33.642740 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.643203 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.653479 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:10:33.653933 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.657435 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:33.657849 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:33.658259 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:33.661158 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:10:33.661666 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.662101 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.662549 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.662996 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.663423 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.663863 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.664291 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.675263 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:33.698980 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.699403 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.707940 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:10:33.711855 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:33.712216 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:33.715200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.715840 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.724821 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:10:33.731752 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:10:33.738886 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.739393 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.739843 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.740592 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.741040 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.741497 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.783900 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:10:33.784249 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.784605 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.785010 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.799091 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.802733 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.803192 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.809591 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:10:33.810139 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.810639 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.811135 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.811658 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.828641 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:10:33.829004 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.829399 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.829879 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.832560 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.832975 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.833338 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.868659 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:10:33.873471 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:10:33.877890 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.880067 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.882179 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:10:33.882573 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.882951 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.891703 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.893588 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:10:33.898070 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.898463 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.898952 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.899347 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.910922 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:10:33.911310 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.911737 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.916407 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:33.922444 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:10:33.922932 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.923399 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.923735 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.924047 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.924590 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.924970 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.925300 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.925745 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.926103 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.926484 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.926917 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.927379 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.927806 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:33.975516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: WSGI app 0 (mountpoint='') ready in 9 seconds on interpreter 0x55774f17c9c0 pid: 80400 (default app) Jun 21 18:10:33.980026 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66bf1a-5c17-4593-8f2b-c0d4e3096db2 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:10:33.980410 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66bf1a-5c17-4593-8f2b-c0d4e3096db2 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:10:33.980757 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66bf1a-5c17-4593-8f2b-c0d4e3096db2 None None] PATH_INFO: `/v3` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:10:33.981963 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:33.982350 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:33.985868 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1/1] 10.222.0.87 () {50 vars in 791 bytes} [Tue Jun 21 18:10:33 2022] GET /identity/v3/ => generated 255 bytes in 10 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:10:34.063577 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:34.064032 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:34.064307 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:34.064788 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:34.065069 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:34.065457 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:34.093388 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:34.093743 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:34.094143 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:34.094523 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:34.095006 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:34.095332 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:34.095581 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:34.108283 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:34.113560 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:34.118661 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:34.118661 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:34.118661 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:34.118661 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:34.118661 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:34.118661 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:10:34.141359 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.141359 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 21 18:10:34.141359 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.141359 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 21 18:10:34.141359 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.141359 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.141359 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.144410 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.161943 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.162568 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.163187 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.163762 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.188662 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:10:34.194139 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:10:34.194650 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.195338 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.195841 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.196418 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.196797 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.197426 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.197952 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.198688 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.199195 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.199697 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.256431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.259760 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:10:34.260321 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.271414 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:10:34.271914 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.272541 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.288014 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:10:34.288519 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.298245 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:10:34.298766 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.299263 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.299741 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.300203 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.300689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.301152 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.301611 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.340461 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.341052 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.350420 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:10:34.358148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.358647 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.369766 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:10:34.377619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:10:34.385177 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.385785 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.386268 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.386759 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.387233 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.387702 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.429801 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:10:34.430296 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.430790 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.431258 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.448077 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.451559 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.452031 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.458067 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:10:34.458565 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.459049 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.459556 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.460039 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.488068 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:10:34.488565 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.489044 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.489536 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.490035 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.490936 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.491309 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.530354 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:10:34.536915 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:10:34.543832 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.547098 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.550228 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:10:34.550700 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.551204 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.565702 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.569127 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:10:34.575966 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.576481 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.576967 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.577429 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.596234 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:10:34.596737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.597195 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.614082 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:10:34.614448 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.614941 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.615728 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.616206 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.616681 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.617171 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.617683 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.618163 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.618629 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.619155 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.619646 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.620298 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.620758 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:10:34.698870 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: WSGI app 0 (mountpoint='') ready in 10 seconds on interpreter 0x55774f17c9c0 pid: 80399 (default app) Jun 21 18:10:45.441027 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd0947-9ff0-47fd-8603-76309eb770e0 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:10:45.441895 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd0947-9ff0-47fd-8603-76309eb770e0 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:10:45.441895 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd0947-9ff0-47fd-8603-76309eb770e0 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:10:45.443843 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 2/2] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:10:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:10:45.457158 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571ecec-5e74-4c24-907c-79bfc12e137f None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:10:45.457783 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571ecec-5e74-4c24-907c-79bfc12e137f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:10:45.458138 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571ecec-5e74-4c24-907c-79bfc12e137f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:10:45.561980 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: INFO dbcounter [None req-d571ecec-5e74-4c24-907c-79bfc12e137f None None] Registered counter for database keystone Jun 21 18:10:45.587559 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG oslo_db.sqlalchemy.engines [None req-d571ecec-5e74-4c24-907c-79bfc12e137f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80399) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 21 18:10:45.596066 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writer thread running {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 21 18:10:45.853699 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG passlib.handlers.bcrypt [None req-d571ecec-5e74-4c24-907c-79bfc12e137f None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80399) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 21 18:10:45.854458 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG passlib.handlers.bcrypt [None req-d571ecec-5e74-4c24-907c-79bfc12e137f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80399) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 21 18:10:45.925907 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d571ecec-5e74-4c24-907c-79bfc12e137f None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:10:45.928616 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d571ecec-5e74-4c24-907c-79bfc12e137f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:10:46.171620 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1/3] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:10:45 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 720 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 21 18:10:46.337754 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd0947-9ff0-47fd-8603-76309eb770e0 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:10:46.427242 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: INFO dbcounter [None req-befd0947-9ff0-47fd-8603-76309eb770e0 None None] Registered counter for database keystone Jun 21 18:10:46.459586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG oslo_db.sqlalchemy.engines [None req-befd0947-9ff0-47fd-8603-76309eb770e0 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80400) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 21 18:10:46.471239 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writer thread running {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 21 18:10:46.720081 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd0947-9ff0-47fd-8603-76309eb770e0 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:10:46.727324 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594cc261-f7b5-4af0-a386-fd811a8459bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:10:46.728465 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594cc261-f7b5-4af0-a386-fd811a8459bc admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:10:46.728931 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594cc261-f7b5-4af0-a386-fd811a8459bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:10:46.729375 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594cc261-f7b5-4af0-a386-fd811a8459bc admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:10:46.760847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594cc261-f7b5-4af0-a386-fd811a8459bc admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:10:46.824434 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: WARNING keystone.common.rbac_enforcer.enforcer [None req-594cc261-f7b5-4af0-a386-fd811a8459bc admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 21 18:10:46.824971 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594cc261-f7b5-4af0-a386-fd811a8459bc admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:10:46.843521 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 3/4] 10.222.0.87 () {58 vars in 1324 bytes} [Tue Jun 21 18:10:46 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 18:10:46.853391 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d571ecec-5e74-4c24-907c-79bfc12e137f None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:10:46.880936 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d571ecec-5e74-4c24-907c-79bfc12e137f None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:10:46.887964 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4475164b-8e7a-47ef-a9a0-bbabf453b736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:10:46.889772 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475164b-8e7a-47ef-a9a0-bbabf453b736 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:10:46.891282 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475164b-8e7a-47ef-a9a0-bbabf453b736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:10:46.891282 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475164b-8e7a-47ef-a9a0-bbabf453b736 admin admin] PATH_INFO: `/v3/projects/58da051d3b4a47fab4c706fb517e9ae8` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:10:46.896799 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4475164b-8e7a-47ef-a9a0-bbabf453b736 admin admin] RBAC: Authorizing `identity:get_project(project_id=58da051d3b4a47fab4c706fb517e9ae8)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:10:46.980666 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: WARNING keystone.common.rbac_enforcer.enforcer [None req-4475164b-8e7a-47ef-a9a0-bbabf453b736 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 21 18:10:46.981011 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4475164b-8e7a-47ef-a9a0-bbabf453b736 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:10:46.987423 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 2/5] 10.222.0.87 () {56 vars in 1207 bytes} [Tue Jun 21 18:10:46 2022] GET /identity/v3/projects/58da051d3b4a47fab4c706fb517e9ae8 => generated 344 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:10:49.907445 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c397f-3503-4ebc-9ac0-24dceeee2053 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:10:49.908274 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c397f-3503-4ebc-9ac0-24dceeee2053 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:10:49.908417 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c397f-3503-4ebc-9ac0-24dceeee2053 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:10:49.913165 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 4/6] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:10:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:10:49.921390 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c62e0f-8edd-4653-874e-53f62f1f09a8 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:10:49.922045 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c62e0f-8edd-4653-874e-53f62f1f09a8 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:10:49.922558 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c62e0f-8edd-4653-874e-53f62f1f09a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:10:49.958730 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-84c62e0f-8edd-4653-874e-53f62f1f09a8 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:10:49.961711 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-84c62e0f-8edd-4653-874e-53f62f1f09a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:10:49.977451 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 3/7] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:10:49 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 21 18:10:50.022347 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1c397f-3503-4ebc-9ac0-24dceeee2053 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:10:50.045482 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1c397f-3503-4ebc-9ac0-24dceeee2053 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:10:50.053299 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72667bc5-033b-4e82-9a94-f5c2c4e20cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:10:50.054554 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72667bc5-033b-4e82-9a94-f5c2c4e20cf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:10:50.055070 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72667bc5-033b-4e82-9a94-f5c2c4e20cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:10:50.056256 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72667bc5-033b-4e82-9a94-f5c2c4e20cf5 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:10:50.074003 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72667bc5-033b-4e82-9a94-f5c2c4e20cf5 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:10:50.077870 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72667bc5-033b-4e82-9a94-f5c2c4e20cf5 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:10:50.093123 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 5/8] 10.222.0.87 () {58 vars in 1324 bytes} [Tue Jun 21 18:10:50 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 18:10:50.104345 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c62e0f-8edd-4653-874e-53f62f1f09a8 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:10:50.125787 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c62e0f-8edd-4653-874e-53f62f1f09a8 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:10:50.133251 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b8ef2-0220-49ab-b897-c3f2d084fc68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:10:50.134910 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b8ef2-0220-49ab-b897-c3f2d084fc68 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:10:50.135404 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b8ef2-0220-49ab-b897-c3f2d084fc68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:10:50.135861 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b8ef2-0220-49ab-b897-c3f2d084fc68 admin admin] PATH_INFO: `/v3/users/dd94cb8910014827bad3f45b205880f7` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:10:50.138614 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7b8ef2-0220-49ab-b897-c3f2d084fc68 admin admin] RBAC: Authorizing `identity:get_user(user_id=dd94cb8910014827bad3f45b205880f7)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:10:50.142046 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7b8ef2-0220-49ab-b897-c3f2d084fc68 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:10:50.145135 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 4/9] 10.222.0.87 () {56 vars in 1198 bytes} [Tue Jun 21 18:10:50 2022] GET /identity/v3/users/dd94cb8910014827bad3f45b205880f7 => generated 247 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:10:53.368880 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6fd4ea-7ad4-428d-a8f1-a76153984b56 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:10:53.369774 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6fd4ea-7ad4-428d-a8f1-a76153984b56 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:10:53.369774 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6fd4ea-7ad4-428d-a8f1-a76153984b56 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:10:53.371617 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 6/10] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:10:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:10:53.379513 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dffceb9-93f2-43d3-8eab-93718645b648 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:10:53.380579 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dffceb9-93f2-43d3-8eab-93718645b648 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:10:53.380579 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dffceb9-93f2-43d3-8eab-93718645b648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:10:53.410868 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-2dffceb9-93f2-43d3-8eab-93718645b648 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:10:53.416732 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2dffceb9-93f2-43d3-8eab-93718645b648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:10:53.581015 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 5/11] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:10:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:10:53.584747 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085017e-f8d9-45c7-9f3b-b54e645ab38e None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:10:53.585351 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085017e-f8d9-45c7-9f3b-b54e645ab38e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:10:53.585808 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085017e-f8d9-45c7-9f3b-b54e645ab38e None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:10:53.589008 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 7/12] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:10:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:10:53.598766 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d93992-bd14-4487-814d-8a9e1a676043 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:10:53.599288 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d93992-bd14-4487-814d-8a9e1a676043 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:10:53.599803 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d93992-bd14-4487-814d-8a9e1a676043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:10:53.615867 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085017e-f8d9-45c7-9f3b-b54e645ab38e None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:10:53.626788 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-32d93992-bd14-4487-814d-8a9e1a676043 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:10:53.629191 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-32d93992-bd14-4487-814d-8a9e1a676043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:10:53.659892 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:10:53.777287 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 6/13] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:10:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:10:53.819711 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d93992-bd14-4487-814d-8a9e1a676043 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:10:53.889556 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085017e-f8d9-45c7-9f3b-b54e645ab38e None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:10:53.941804 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc4b0a6-a755-4546-9043-5777b6b14af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:10:53.943091 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4b0a6-a755-4546-9043-5777b6b14af4 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:10:53.943547 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4b0a6-a755-4546-9043-5777b6b14af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:10:53.944013 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4b0a6-a755-4546-9043-5777b6b14af4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:10:53.975119 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc4b0a6-a755-4546-9043-5777b6b14af4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:10:53.996766 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-3fc4b0a6-a755-4546-9043-5777b6b14af4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:10:53.996766 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:10:53.996766 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:10:53.996766 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:10:53.996766 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:10:53.996766 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:10:53.996766 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:10:53.996766 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:10:53.996766 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:10:53.996766 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:10:53.996766 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:10:53.996766 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:10:53.996766 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:10:53.996766 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:10:53.996766 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:10:53.996766 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:10:53.998847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:10:53.998847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:10:53.998847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:10:53.998847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:10:53.998847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:10:53.998847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:10:53.998847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:10:53.998847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:10:53.998847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:10:53.998847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:10:53.998847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:10:53.998847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:10:53.998847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:10:53.998847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:10:53.998847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:10:53.998847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:10:54.000398 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:10:54.000398 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:10:54.000398 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:10:54.000398 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:10:54.000398 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:10:54.000398 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:10:54.000398 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:10:54.000398 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:10:54.000398 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:10:54.000398 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:10:54.000398 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:10:54.000398 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:10:54.005910 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 8/14] 10.222.0.87 () {56 vars in 1096 bytes} [Tue Jun 21 18:10:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 391 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:10:54.008496 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc4b0a6-a755-4546-9043-5777b6b14af4 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:10:54.038639 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d93992-bd14-4487-814d-8a9e1a676043 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:10:54.081529 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:10:54.120120 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82847217-9496-4a56-9c26-09cb3cc878c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:10:54.120920 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82847217-9496-4a56-9c26-09cb3cc878c5 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:10:54.121354 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82847217-9496-4a56-9c26-09cb3cc878c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:10:54.121798 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82847217-9496-4a56-9c26-09cb3cc878c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:10:54.157574 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:10:54.234177 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc4b0a6-a755-4546-9043-5777b6b14af4 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:10:54.248180 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82847217-9496-4a56-9c26-09cb3cc878c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:10:54.250619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82847217-9496-4a56-9c26-09cb3cc878c5 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:10:54.303562 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70818b7-b706-48e3-afd1-c3fee59839e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:10:54.304721 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70818b7-b706-48e3-afd1-c3fee59839e1 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:10:54.305787 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70818b7-b706-48e3-afd1-c3fee59839e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:10:54.306299 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70818b7-b706-48e3-afd1-c3fee59839e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:10:54.307799 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70818b7-b706-48e3-afd1-c3fee59839e1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:10:54.308344 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70818b7-b706-48e3-afd1-c3fee59839e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:10:54.311026 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70818b7-b706-48e3-afd1-c3fee59839e1 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:10:54.334113 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 9/15] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:10:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:10:54.343431 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70818b7-b706-48e3-afd1-c3fee59839e1 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:10:54.445010 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 7/16] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:10:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 625 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 21 18:10:54.451897 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82847217-9496-4a56-9c26-09cb3cc878c5 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:10:54.529071 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:10:54.537547 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:10:54.541465 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70818b7-b706-48e3-afd1-c3fee59839e1 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:10:54.600170 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9b085e-b17c-4f60-9ad2-83f176b77e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:10:54.602296 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9b085e-b17c-4f60-9ad2-83f176b77e17 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:10:54.602296 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9b085e-b17c-4f60-9ad2-83f176b77e17 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:10:54.602296 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9b085e-b17c-4f60-9ad2-83f176b77e17 None admin] PATH_INFO: `/v3/users/dd94cb8910014827bad3f45b205880f7` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:10:54.604672 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9b085e-b17c-4f60-9ad2-83f176b77e17 None admin] RBAC: Authorizing `identity:get_user(user_id=dd94cb8910014827bad3f45b205880f7)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:10:54.607151 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9b085e-b17c-4f60-9ad2-83f176b77e17 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:10:54.610317 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 10/17] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:10:54 2022] GET /identity/v3/users/dd94cb8910014827bad3f45b205880f7 => generated 247 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:10:54.619759 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9b085e-b17c-4f60-9ad2-83f176b77e17 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:10:54.630817 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82847217-9496-4a56-9c26-09cb3cc878c5 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:10:54.691463 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15570b13-6be0-4801-84a9-2b3d054dbf74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:10:54.694116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15570b13-6be0-4801-84a9-2b3d054dbf74 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:10:54.694717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15570b13-6be0-4801-84a9-2b3d054dbf74 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:10:54.695275 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15570b13-6be0-4801-84a9-2b3d054dbf74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:10:54.718384 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15570b13-6be0-4801-84a9-2b3d054dbf74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:10:54.721475 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15570b13-6be0-4801-84a9-2b3d054dbf74 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:10:54.766875 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-15570b13-6be0-4801-84a9-2b3d054dbf74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:10:54.766875 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:10:54.766875 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:10:54.766875 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:10:54.766875 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:10:54.766875 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:10:54.766875 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:10:54.766875 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:10:54.766875 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:10:54.766875 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:10:54.766875 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:10:54.766875 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:10:54.766875 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:10:54.766875 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:10:54.766875 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:10:54.768362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:10:54.768362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:10:54.768362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:10:54.768362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:10:54.768362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:10:54.768362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:10:54.768362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:10:54.768362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:10:54.768362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:10:54.768362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:10:54.768362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:10:54.768362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:10:54.768362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:10:54.768362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:10:54.768362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:10:54.768362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:10:54.768362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:10:54.769950 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:10:54.769950 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:10:54.769950 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:10:54.769950 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:10:54.769950 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 8/18] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:10:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:10:54.779886 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15570b13-6be0-4801-84a9-2b3d054dbf74 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:10:54.843991 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9b085e-b17c-4f60-9ad2-83f176b77e17 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:10:54.912434 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98f1e1-1640-4411-a741-b6f9d71adc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:10:54.913732 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98f1e1-1640-4411-a741-b6f9d71adc45 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:10:54.914227 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98f1e1-1640-4411-a741-b6f9d71adc45 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:10:54.914721 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98f1e1-1640-4411-a741-b6f9d71adc45 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:10:54.918705 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d98f1e1-1640-4411-a741-b6f9d71adc45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:10:54.920301 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d98f1e1-1640-4411-a741-b6f9d71adc45 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:10:54.923076 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 11/19] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:10:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:10:54.933245 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98f1e1-1640-4411-a741-b6f9d71adc45 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:10:54.985773 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15570b13-6be0-4801-84a9-2b3d054dbf74 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:10:54.989228 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:10:54.994737 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:10:55.058157 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab774c-e2a9-4769-a8a7-bad27f27f5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:10:55.059208 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab774c-e2a9-4769-a8a7-bad27f27f5b2 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:10:55.066558 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab774c-e2a9-4769-a8a7-bad27f27f5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:10:55.067179 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab774c-e2a9-4769-a8a7-bad27f27f5b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:10:55.068475 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cab774c-e2a9-4769-a8a7-bad27f27f5b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:10:55.068966 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cab774c-e2a9-4769-a8a7-bad27f27f5b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:10:55.071607 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cab774c-e2a9-4769-a8a7-bad27f27f5b2 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:10:55.122856 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 9/20] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:10:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:10:55.140864 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98f1e1-1640-4411-a741-b6f9d71adc45 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:10:55.222195 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4664081-3959-417c-a913-a29686076771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:10:55.223373 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4664081-3959-417c-a913-a29686076771 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:10:55.223893 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4664081-3959-417c-a913-a29686076771 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:10:55.224410 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4664081-3959-417c-a913-a29686076771 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:10:55.226245 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4664081-3959-417c-a913-a29686076771 None admin] RBAC: Adding query filter params (role.id=98f0943a610f4a9e9f8de54f6e9f717d, scope.domain.id=default, user.id=dd94cb8910014827bad3f45b205880f7) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:10:55.226841 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4664081-3959-417c-a913-a29686076771 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:10:55.231273 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4664081-3959-417c-a913-a29686076771 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:10:55.252287 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 12/21] 10.222.0.87 () {56 vars in 1322 bytes} [Tue Jun 21 18:10:54 2022] GET /identity/v3/role_assignments?user.id=dd94cb8910014827bad3f45b205880f7&scope.domain.id=default&role.id=98f0943a610f4a9e9f8de54f6e9f717d => generated 233 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:10:59.250858 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7791576b-e834-4881-9ca5-60720dcb3fb6 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:10:59.252464 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7791576b-e834-4881-9ca5-60720dcb3fb6 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:10:59.253181 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7791576b-e834-4881-9ca5-60720dcb3fb6 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:10:59.256120 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 10/22] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:10:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:10:59.265674 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def09c7-aab3-4503-864d-984a3b888183 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:10:59.267016 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def09c7-aab3-4503-864d-984a3b888183 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:10:59.267790 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def09c7-aab3-4503-864d-984a3b888183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:10:59.297977 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG passlib.handlers.bcrypt [None req-4def09c7-aab3-4503-864d-984a3b888183 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80400) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 21 18:10:59.298863 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG passlib.handlers.bcrypt [None req-4def09c7-aab3-4503-864d-984a3b888183 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80400) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 21 18:10:59.353317 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-4def09c7-aab3-4503-864d-984a3b888183 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:10:59.355380 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4def09c7-aab3-4503-864d-984a3b888183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:10:59.392933 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eda578-1cd5-4ff9-9015-2705af1e8892 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:10:59.393906 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eda578-1cd5-4ff9-9015-2705af1e8892 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:10:59.394691 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eda578-1cd5-4ff9-9015-2705af1e8892 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:10:59.399137 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 11/23] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:10:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:10:59.406386 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c013a1-b50c-4744-a416-74aaafc4a22d None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:10:59.407341 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c013a1-b50c-4744-a416-74aaafc4a22d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:10:59.408108 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c013a1-b50c-4744-a416-74aaafc4a22d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:10:59.463092 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-68c013a1-b50c-4744-a416-74aaafc4a22d None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:10:59.464759 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-68c013a1-b50c-4744-a416-74aaafc4a22d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:10:59.531007 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 13/24] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:10:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:10:59.572675 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4def09c7-aab3-4503-864d-984a3b888183 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:10:59.611254 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 12/25] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:10:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:10:59.650298 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:10:59.688199 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c013a1-b50c-4744-a416-74aaafc4a22d None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:10:59.847322 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4def09c7-aab3-4503-864d-984a3b888183 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:10:59.908455 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d80a9db-4181-4fbe-9577-25227de44df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:10:59.910900 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d80a9db-4181-4fbe-9577-25227de44df1 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:10:59.911655 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d80a9db-4181-4fbe-9577-25227de44df1 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:10:59.912562 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d80a9db-4181-4fbe-9577-25227de44df1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:10:59.951085 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d80a9db-4181-4fbe-9577-25227de44df1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:10:59.954056 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d80a9db-4181-4fbe-9577-25227de44df1 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:10:59.985701 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-7d80a9db-4181-4fbe-9577-25227de44df1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:10:59.985701 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:10:59.985701 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:10:59.985701 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:10:59.985701 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:10:59.985701 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:10:59.985701 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:10:59.985701 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:10:59.985701 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:10:59.985701 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:10:59.985701 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:10:59.985701 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:10:59.985701 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:10:59.985701 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:10:59.985701 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:10:59.987231 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:10:59.987231 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:10:59.987231 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:10:59.987231 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:10:59.987231 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:10:59.987231 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:10:59.987231 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:10:59.987231 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:10:59.987231 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:10:59.987231 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:10:59.987231 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:10:59.987231 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:10:59.987231 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:10:59.987231 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:10:59.987231 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:10:59.987231 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:10:59.987231 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:10:59.989012 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:10:59.989012 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:10:59.989012 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:10:59.989012 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:10:59.992633 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 14/26] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:10:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 421 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:10:59.995479 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d9ed7-7fa3-4a7c-882c-fed7ccc04a72 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:10:59.996359 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d9ed7-7fa3-4a7c-882c-fed7ccc04a72 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:10:59.996887 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d9ed7-7fa3-4a7c-882c-fed7ccc04a72 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:10:59.999183 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 15/27] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:10:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:11:00.000298 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2d9ed7-7fa3-4a7c-882c-fed7ccc04a72 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:00.041438 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:00.061680 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c013a1-b50c-4744-a416-74aaafc4a22d None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:00.138640 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:00.169946 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2d9ed7-7fa3-4a7c-882c-fed7ccc04a72 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:00.172376 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe7e6d7-5a04-4ede-a5fe-5a7378d7a47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:00.174165 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7e6d7-5a04-4ede-a5fe-5a7378d7a47a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:00.174505 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7e6d7-5a04-4ede-a5fe-5a7378d7a47a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:00.175000 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7e6d7-5a04-4ede-a5fe-5a7378d7a47a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:00.199306 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe7e6d7-5a04-4ede-a5fe-5a7378d7a47a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:11:00.219093 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-2fe7e6d7-5a04-4ede-a5fe-5a7378d7a47a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:11:00.219093 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:11:00.219093 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:11:00.219093 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:11:00.219093 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:11:00.219093 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:11:00.219093 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:11:00.219093 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:11:00.219093 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:11:00.219093 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:11:00.219093 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:11:00.219093 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:11:00.219093 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:11:00.219093 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:11:00.219093 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:11:00.219093 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:11:00.220844 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:11:00.220844 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:11:00.220844 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:11:00.220844 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:11:00.220844 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:11:00.220844 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:11:00.220844 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:11:00.220844 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:11:00.220844 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:11:00.220844 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:11:00.220844 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:11:00.220844 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:11:00.220844 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:11:00.220844 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:11:00.220844 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:11:00.220844 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:11:00.222382 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:11:00.222382 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:11:00.222382 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:11:00.222382 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:11:00.222382 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:11:00.222382 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:11:00.222382 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:11:00.222382 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:11:00.222382 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:11:00.222382 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:11:00.222382 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:11:00.222382 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:11:00.222382 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 13/28] 10.222.0.87 () {56 vars in 1096 bytes} [Tue Jun 21 18:10:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 534 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:11:00.224885 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5494a9-7610-4e37-94e7-54e0e5ee62f9 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:00.224885 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5494a9-7610-4e37-94e7-54e0e5ee62f9 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:00.224885 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5494a9-7610-4e37-94e7-54e0e5ee62f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:00.251396 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439e69d4-e62c-46c7-b40a-791f498e0871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:00.253027 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e69d4-e62c-46c7-b40a-791f498e0871 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:00.253835 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e69d4-e62c-46c7-b40a-791f498e0871 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:00.254635 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e69d4-e62c-46c7-b40a-791f498e0871 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:00.256555 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439e69d4-e62c-46c7-b40a-791f498e0871 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:00.257114 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-ee5494a9-7610-4e37-94e7-54e0e5ee62f9 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:11:00.258562 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439e69d4-e62c-46c7-b40a-791f498e0871 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:00.259488 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ee5494a9-7610-4e37-94e7-54e0e5ee62f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:11:00.261928 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439e69d4-e62c-46c7-b40a-791f498e0871 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:00.300026 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 16/29] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:10:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:00.303884 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579f445-93a0-49bd-9b75-c24f569d9317 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:00.304962 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579f445-93a0-49bd-9b75-c24f569d9317 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:00.305827 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579f445-93a0-49bd-9b75-c24f569d9317 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:00.309671 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 17/30] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:11:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:11:00.311572 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7579f445-93a0-49bd-9b75-c24f569d9317 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:00.418774 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 14/31] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:11:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:11:00.420592 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5494a9-7610-4e37-94e7-54e0e5ee62f9 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:00.476694 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7579f445-93a0-49bd-9b75-c24f569d9317 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:00.531701 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb805ff-6ab7-4e13-9386-3ebda46ecbec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:00.533266 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb805ff-6ab7-4e13-9386-3ebda46ecbec None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:00.534133 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb805ff-6ab7-4e13-9386-3ebda46ecbec None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:00.534843 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb805ff-6ab7-4e13-9386-3ebda46ecbec None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:00.536369 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb805ff-6ab7-4e13-9386-3ebda46ecbec None admin] RBAC: Adding query filter params (name=admin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:00.537112 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb805ff-6ab7-4e13-9386-3ebda46ecbec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:00.540629 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb805ff-6ab7-4e13-9386-3ebda46ecbec None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:00.547889 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:00.557449 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 18/32] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:11:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:00.559686 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5123f-7555-46b2-82a2-a72f9c5c3049 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:00.560260 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5123f-7555-46b2-82a2-a72f9c5c3049 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:00.560862 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5123f-7555-46b2-82a2-a72f9c5c3049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:00.585310 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-ffa5123f-7555-46b2-82a2-a72f9c5c3049 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:11:00.586359 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:00.590060 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ffa5123f-7555-46b2-82a2-a72f9c5c3049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:11:00.614701 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5494a9-7610-4e37-94e7-54e0e5ee62f9 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:00.683649 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bee7fd4-da4a-404b-8fb8-e66d6ed7ed09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:00.684525 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee7fd4-da4a-404b-8fb8-e66d6ed7ed09 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:00.684841 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee7fd4-da4a-404b-8fb8-e66d6ed7ed09 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:00.685238 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee7fd4-da4a-404b-8fb8-e66d6ed7ed09 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:00.686676 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bee7fd4-da4a-404b-8fb8-e66d6ed7ed09 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:00.689691 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bee7fd4-da4a-404b-8fb8-e66d6ed7ed09 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:00.738646 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 19/33] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:11:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:11:00.740595 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa5123f-7555-46b2-82a2-a72f9c5c3049 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:00.916477 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 15/34] 10.222.0.87 () {60 vars in 1141 bytes} [Tue Jun 21 18:11:00 2022] POST /identity/v3/projects => generated 301 bytes in 497 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:11:00.917997 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bee7fd4-da4a-404b-8fb8-e66d6ed7ed09 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:00.975527 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa5123f-7555-46b2-82a2-a72f9c5c3049 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:01.054141 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:01.067060 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3c374e-a4e2-40e7-910b-8cef63ab9df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:01.068401 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3c374e-a4e2-40e7-910b-8cef63ab9df7 None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:01.068947 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3c374e-a4e2-40e7-910b-8cef63ab9df7 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:01.069354 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3c374e-a4e2-40e7-910b-8cef63ab9df7 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:01.070558 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3c374e-a4e2-40e7-910b-8cef63ab9df7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:01.072323 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3c374e-a4e2-40e7-910b-8cef63ab9df7 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:01.211225 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bee7fd4-da4a-404b-8fb8-e66d6ed7ed09 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:01.266593 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 20/35] 10.222.0.87 () {60 vars in 1131 bytes} [Tue Jun 21 18:11:00 2022] POST /identity/v3/roles => generated 219 bytes in 527 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:11:01.271022 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b14e25-d830-4bfd-9743-c434b0d50f95 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:01.271879 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b14e25-d830-4bfd-9743-c434b0d50f95 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:01.272621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b14e25-d830-4bfd-9743-c434b0d50f95 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:01.275792 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 21/36] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:11:01 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:11:01.277594 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b14e25-d830-4bfd-9743-c434b0d50f95 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:01.303481 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f306246-4412-48f2-8b06-2af96db0e1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:01.306905 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f306246-4412-48f2-8b06-2af96db0e1ba None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:01.307671 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f306246-4412-48f2-8b06-2af96db0e1ba None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:01.308347 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f306246-4412-48f2-8b06-2af96db0e1ba None admin] PATH_INFO: `/v3/users/dd94cb8910014827bad3f45b205880f7` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:01.311461 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f306246-4412-48f2-8b06-2af96db0e1ba None admin] RBAC: Authorizing `identity:get_user(user_id=dd94cb8910014827bad3f45b205880f7)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:01.315096 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f306246-4412-48f2-8b06-2af96db0e1ba None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:01.319135 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 16/37] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:11:00 2022] GET /identity/v3/users/dd94cb8910014827bad3f45b205880f7 => generated 247 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:01.322139 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957a4cf-4f72-4733-bc5a-5aaf0c86ed56 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:01.322945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957a4cf-4f72-4733-bc5a-5aaf0c86ed56 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:01.323629 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957a4cf-4f72-4733-bc5a-5aaf0c86ed56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:01.334997 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:01.533785 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-2957a4cf-4f72-4733-bc5a-5aaf0c86ed56 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:11:01.535171 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b14e25-d830-4bfd-9743-c434b0d50f95 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:01.536903 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2957a4cf-4f72-4733-bc5a-5aaf0c86ed56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:11:01.625145 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d93bbde-967a-4575-a921-984c9eedf9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:01.626458 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d93bbde-967a-4575-a921-984c9eedf9cb None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:01.626963 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d93bbde-967a-4575-a921-984c9eedf9cb None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:01.627469 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d93bbde-967a-4575-a921-984c9eedf9cb None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:01.629011 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d93bbde-967a-4575-a921-984c9eedf9cb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:01.632111 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d93bbde-967a-4575-a921-984c9eedf9cb None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:01.663067 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 22/38] 10.222.0.87 () {60 vars in 1131 bytes} [Tue Jun 21 18:11:01 2022] POST /identity/v3/roles => generated 225 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:11:01.665396 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d93bbde-967a-4575-a921-984c9eedf9cb None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:01.728567 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 17/39] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:11:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:11:01.728871 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35760f36-e12d-4db9-a597-846942a6da30 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:01.729147 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35760f36-e12d-4db9-a597-846942a6da30 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:01.729828 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35760f36-e12d-4db9-a597-846942a6da30 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:01.732517 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 18/40] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:11:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:11:01.745177 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf09972-b32e-46d9-a34c-754d0ff3bd90 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:01.745730 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf09972-b32e-46d9-a34c-754d0ff3bd90 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:01.746582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf09972-b32e-46d9-a34c-754d0ff3bd90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:01.781286 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3bf09972-b32e-46d9-a34c-754d0ff3bd90 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:11:01.784150 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3bf09972-b32e-46d9-a34c-754d0ff3bd90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:11:01.796864 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:01.928988 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 19/41] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:11:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:11:01.931635 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf09972-b32e-46d9-a34c-754d0ff3bd90 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:01.939533 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d93bbde-967a-4575-a921-984c9eedf9cb None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:02.019188 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4143da63-75d6-4577-9d5c-457901ab7a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:02.020505 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4143da63-75d6-4577-9d5c-457901ab7a42 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:02.023405 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4143da63-75d6-4577-9d5c-457901ab7a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:02.023405 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4143da63-75d6-4577-9d5c-457901ab7a42 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:02.026546 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4143da63-75d6-4577-9d5c-457901ab7a42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:02.028140 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4143da63-75d6-4577-9d5c-457901ab7a42 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:02.031369 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 23/42] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:11:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:02.032174 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4143da63-75d6-4577-9d5c-457901ab7a42 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:02.086496 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:02.165664 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf09972-b32e-46d9-a34c-754d0ff3bd90 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:02.215895 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9855cd9d-27bb-41c1-a6f9-f94ce0e25ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:02.217050 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9855cd9d-27bb-41c1-a6f9-f94ce0e25ba8 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:02.217522 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9855cd9d-27bb-41c1-a6f9-f94ce0e25ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:02.218045 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9855cd9d-27bb-41c1-a6f9-f94ce0e25ba8 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:02.219437 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9855cd9d-27bb-41c1-a6f9-f94ce0e25ba8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:02.222184 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9855cd9d-27bb-41c1-a6f9-f94ce0e25ba8 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:02.244536 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4143da63-75d6-4577-9d5c-457901ab7a42 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:02.250506 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 20/43] 10.222.0.87 () {60 vars in 1131 bytes} [Tue Jun 21 18:11:01 2022] POST /identity/v3/roles => generated 223 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:11:02.251117 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9855cd9d-27bb-41c1-a6f9-f94ce0e25ba8 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:02.302383 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:02.346795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54ffa92-5d10-475a-a44d-df0926a24d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:02.348557 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ffa92-5d10-475a-a44d-df0926a24d87 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:02.349087 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ffa92-5d10-475a-a44d-df0926a24d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:02.349495 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ffa92-5d10-475a-a44d-df0926a24d87 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:02.351648 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54ffa92-5d10-475a-a44d-df0926a24d87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:02.354537 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54ffa92-5d10-475a-a44d-df0926a24d87 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:02.358005 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 24/44] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:11:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:02.366313 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54ffa92-5d10-475a-a44d-df0926a24d87 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:02.505363 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9855cd9d-27bb-41c1-a6f9-f94ce0e25ba8 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:02.570347 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:02.604405 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7a0a5e-1011-489f-ada3-8568ccff2f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:02.606805 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a0a5e-1011-489f-ada3-8568ccff2f4f None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:02.607266 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a0a5e-1011-489f-ada3-8568ccff2f4f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:02.607773 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a0a5e-1011-489f-ada3-8568ccff2f4f None admin] PATH_INFO: `/v3/domains/default/users/dd94cb8910014827bad3f45b205880f7/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:02.611197 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7a0a5e-1011-489f-ada3-8568ccff2f4f None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=dd94cb8910014827bad3f45b205880f7, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:02.613845 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7a0a5e-1011-489f-ada3-8568ccff2f4f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:02.621366 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54ffa92-5d10-475a-a44d-df0926a24d87 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:02.637039 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 21/45] 10.222.0.87 () {58 vars in 1361 bytes} [Tue Jun 21 18:11:02 2022] PUT /identity/v3/domains/default/users/dd94cb8910014827bad3f45b205880f7/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:11:02.688295 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d16773-08a0-4b6e-bea9-1b06b0a32f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:02.689340 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d16773-08a0-4b6e-bea9-1b06b0a32f03 None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:02.689676 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d16773-08a0-4b6e-bea9-1b06b0a32f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:02.690125 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d16773-08a0-4b6e-bea9-1b06b0a32f03 None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:02.690891 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d16773-08a0-4b6e-bea9-1b06b0a32f03 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:02.692561 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d16773-08a0-4b6e-bea9-1b06b0a32f03 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:02.739662 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 25/46] 10.222.0.87 () {60 vars in 1141 bytes} [Tue Jun 21 18:11:02 2022] POST /identity/v3/projects => generated 312 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:11:05.450735 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de17377-ba3c-4496-bd70-5fe323b7bde5 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:05.451316 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de17377-ba3c-4496-bd70-5fe323b7bde5 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:05.451740 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de17377-ba3c-4496-bd70-5fe323b7bde5 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:05.454475 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 22/47] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:11:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:11:05.460864 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc71b71-07a4-4db4-a567-d1696c50d06d None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:05.461676 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc71b71-07a4-4db4-a567-d1696c50d06d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:05.461676 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc71b71-07a4-4db4-a567-d1696c50d06d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:05.483056 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf42bc-1d64-4d51-b69c-982c6e68aff8 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:05.483358 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf42bc-1d64-4d51-b69c-982c6e68aff8 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:05.483868 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf42bc-1d64-4d51-b69c-982c6e68aff8 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:05.484042 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-4fc71b71-07a4-4db4-a567-d1696c50d06d None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:11:05.485547 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4fc71b71-07a4-4db4-a567-d1696c50d06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:11:05.485771 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 23/48] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:11:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:11:05.494378 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444b691-9bef-4688-8da1-4b49464152b8 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:05.494867 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444b691-9bef-4688-8da1-4b49464152b8 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:05.495977 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444b691-9bef-4688-8da1-4b49464152b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:05.525879 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3444b691-9bef-4688-8da1-4b49464152b8 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:11:05.527996 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3444b691-9bef-4688-8da1-4b49464152b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:11:05.530588 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:05.591138 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 26/49] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:11:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:11:05.622803 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc71b71-07a4-4db4-a567-d1696c50d06d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:05.655380 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 24/50] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:11:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:11:05.686418 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3444b691-9bef-4688-8da1-4b49464152b8 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:05.787813 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc71b71-07a4-4db4-a567-d1696c50d06d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:05.845014 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599cc8a2-105d-4778-8c6f-1c2566a98c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:05.846455 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599cc8a2-105d-4778-8c6f-1c2566a98c6a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:05.846957 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599cc8a2-105d-4778-8c6f-1c2566a98c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:05.847435 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599cc8a2-105d-4778-8c6f-1c2566a98c6a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:05.849892 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599cc8a2-105d-4778-8c6f-1c2566a98c6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:05.853015 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599cc8a2-105d-4778-8c6f-1c2566a98c6a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:05.856381 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 27/51] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:11:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:11:05.863500 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599cc8a2-105d-4778-8c6f-1c2566a98c6a None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:05.880161 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3444b691-9bef-4688-8da1-4b49464152b8 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:05.902199 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:05.912325 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:05.944922 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9627b6b2-6280-482c-8503-bff945ddeb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:05.946062 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9627b6b2-6280-482c-8503-bff945ddeb75 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:05.946536 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9627b6b2-6280-482c-8503-bff945ddeb75 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:05.946923 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9627b6b2-6280-482c-8503-bff945ddeb75 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:05.962446 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9627b6b2-6280-482c-8503-bff945ddeb75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:11:05.971926 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-9627b6b2-6280-482c-8503-bff945ddeb75 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:11:05.971926 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:11:05.971926 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:11:05.971926 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:11:05.971926 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:11:05.971926 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:11:05.971926 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:11:05.971926 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:11:05.971926 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:11:05.971926 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:11:05.971926 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:11:05.971926 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:11:05.971926 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:11:05.971926 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:11:05.971926 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:11:05.971926 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:11:05.973577 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:11:05.973577 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:11:05.973577 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:11:05.973577 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:11:05.973577 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:11:05.973577 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:11:05.973577 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:11:05.973577 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:11:05.973577 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:11:05.973577 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:11:05.973577 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:11:05.973577 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:11:05.973577 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:11:05.973577 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:11:05.973577 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:11:05.973577 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:11:05.975004 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:11:05.975004 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:11:05.975004 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:11:05.975004 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:11:05.975004 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:11:05.975004 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:11:05.975004 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:11:05.975004 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:11:05.975004 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:11:05.975004 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:11:05.975004 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:11:05.975004 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:11:05.975004 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 25/52] 10.222.0.87 () {56 vars in 1096 bytes} [Tue Jun 21 18:11:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:11:05.982611 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9627b6b2-6280-482c-8503-bff945ddeb75 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:06.026087 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599cc8a2-105d-4778-8c6f-1c2566a98c6a None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:06.179669 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9627b6b2-6280-482c-8503-bff945ddeb75 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:06.230641 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0efa87-0b9d-4d42-acfa-0959dd46a900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:06.233770 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0efa87-0b9d-4d42-acfa-0959dd46a900 None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:06.233770 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0efa87-0b9d-4d42-acfa-0959dd46a900 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:06.233770 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0efa87-0b9d-4d42-acfa-0959dd46a900 None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:06.234285 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0efa87-0b9d-4d42-acfa-0959dd46a900 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:06.235844 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0efa87-0b9d-4d42-acfa-0959dd46a900 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:06.239294 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644b7e08-5883-4422-b468-74ac851cd8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:06.240371 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b7e08-5883-4422-b468-74ac851cd8b7 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:06.240797 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b7e08-5883-4422-b468-74ac851cd8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:06.241301 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b7e08-5883-4422-b468-74ac851cd8b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:06.244040 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644b7e08-5883-4422-b468-74ac851cd8b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:06.245050 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644b7e08-5883-4422-b468-74ac851cd8b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:06.248040 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644b7e08-5883-4422-b468-74ac851cd8b7 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:06.271231 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 26/53] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:11:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:11:06.281844 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644b7e08-5883-4422-b468-74ac851cd8b7 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:06.283314 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 28/54] 10.222.0.87 () {60 vars in 1140 bytes} [Tue Jun 21 18:11:05 2022] POST /identity/v3/projects => generated 298 bytes in 421 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:11:06.365455 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:06.533252 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644b7e08-5883-4422-b468-74ac851cd8b7 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:06.598310 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e274ac-b146-4d29-b41d-a0e023d7a663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:06.599893 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e274ac-b146-4d29-b41d-a0e023d7a663 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:06.600388 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e274ac-b146-4d29-b41d-a0e023d7a663 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:06.600943 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e274ac-b146-4d29-b41d-a0e023d7a663 None admin] PATH_INFO: `/v3/users/dd94cb8910014827bad3f45b205880f7` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:06.603608 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e274ac-b146-4d29-b41d-a0e023d7a663 None admin] RBAC: Authorizing `identity:get_user(user_id=dd94cb8910014827bad3f45b205880f7)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:06.606360 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e274ac-b146-4d29-b41d-a0e023d7a663 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:06.608726 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 27/55] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:11:06 2022] GET /identity/v3/users/dd94cb8910014827bad3f45b205880f7 => generated 247 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:11:06.617941 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0efa87-0b9d-4d42-acfa-0959dd46a900 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:06.803263 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:06.835114 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0efa87-0b9d-4d42-acfa-0959dd46a900 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:06.910678 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b098a3-0024-4653-a11d-0ac75cc8e808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:06.912122 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b098a3-0024-4653-a11d-0ac75cc8e808 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:06.912603 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b098a3-0024-4653-a11d-0ac75cc8e808 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:06.913296 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b098a3-0024-4653-a11d-0ac75cc8e808 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:06.915885 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b098a3-0024-4653-a11d-0ac75cc8e808 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:06.919106 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b098a3-0024-4653-a11d-0ac75cc8e808 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:06.922719 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 29/56] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:11:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:06.932590 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e274ac-b146-4d29-b41d-a0e023d7a663 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:07.085904 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e274ac-b146-4d29-b41d-a0e023d7a663 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:07.090586 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:07.148480 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cf1f72-1aaf-44dc-b401-b76b3022d171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:07.149619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf1f72-1aaf-44dc-b401-b76b3022d171 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:07.150110 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf1f72-1aaf-44dc-b401-b76b3022d171 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:07.150591 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf1f72-1aaf-44dc-b401-b76b3022d171 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:07.152216 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cf1f72-1aaf-44dc-b401-b76b3022d171 None admin] RBAC: Adding query filter params (role.id=98f0943a610f4a9e9f8de54f6e9f717d, scope.domain.id=default, user.id=dd94cb8910014827bad3f45b205880f7) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:07.152736 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cf1f72-1aaf-44dc-b401-b76b3022d171 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:07.155374 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cf1f72-1aaf-44dc-b401-b76b3022d171 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:07.165175 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 28/57] 10.222.0.87 () {56 vars in 1322 bytes} [Tue Jun 21 18:11:06 2022] GET /identity/v3/role_assignments?user.id=dd94cb8910014827bad3f45b205880f7&scope.domain.id=default&role.id=98f0943a610f4a9e9f8de54f6e9f717d => generated 530 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:08.905893 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0956a18-16b7-4641-b21b-40f0f9c70d81 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:08.906757 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0956a18-16b7-4641-b21b-40f0f9c70d81 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:08.906757 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0956a18-16b7-4641-b21b-40f0f9c70d81 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:08.908974 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 30/58] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:11:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:11:08.920666 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62486dc4-07c6-4476-8537-c9a3d3e55667 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:08.921338 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62486dc4-07c6-4476-8537-c9a3d3e55667 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:08.921563 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62486dc4-07c6-4476-8537-c9a3d3e55667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:08.950892 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-62486dc4-07c6-4476-8537-c9a3d3e55667 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:11:08.953007 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-62486dc4-07c6-4476-8537-c9a3d3e55667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:11:09.073586 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 29/59] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:11:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:11:09.100824 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0956a18-16b7-4641-b21b-40f0f9c70d81 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:09.257333 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0956a18-16b7-4641-b21b-40f0f9c70d81 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:09.318847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f84417-a9b8-47c1-b649-fb1a3628f955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:09.320276 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f84417-a9b8-47c1-b649-fb1a3628f955 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:09.320777 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f84417-a9b8-47c1-b649-fb1a3628f955 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:09.321317 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f84417-a9b8-47c1-b649-fb1a3628f955 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:09.323310 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f84417-a9b8-47c1-b649-fb1a3628f955 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:09.326289 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f84417-a9b8-47c1-b649-fb1a3628f955 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:09.329449 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 31/60] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:11:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:09.338760 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62486dc4-07c6-4476-8537-c9a3d3e55667 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:09.536899 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:09.542582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62486dc4-07c6-4476-8537-c9a3d3e55667 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:09.596255 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4a825e-25df-4489-bec5-cd999def3847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:09.596906 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a825e-25df-4489-bec5-cd999def3847 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:09.597239 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a825e-25df-4489-bec5-cd999def3847 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:09.597563 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a825e-25df-4489-bec5-cd999def3847 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:09.598417 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4a825e-25df-4489-bec5-cd999def3847 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:09.600474 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4a825e-25df-4489-bec5-cd999def3847 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:09.637929 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 30/61] 10.222.0.87 () {60 vars in 1132 bytes} [Tue Jun 21 18:11:09 2022] POST /identity/v3/users => generated 275 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:11:14.491846 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7fe399-a4c3-4107-a127-9434f4c3775b None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:14.492698 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7fe399-a4c3-4107-a127-9434f4c3775b None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:14.492856 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7fe399-a4c3-4107-a127-9434f4c3775b None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:14.496032 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 32/62] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:11:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:11:14.506625 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923a714-a2ca-4504-8f84-b629ea214bdc None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:14.506625 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923a714-a2ca-4504-8f84-b629ea214bdc None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:14.506625 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923a714-a2ca-4504-8f84-b629ea214bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:14.542993 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-e923a714-a2ca-4504-8f84-b629ea214bdc None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:11:14.545819 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e923a714-a2ca-4504-8f84-b629ea214bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:11:14.601413 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab843578-679a-4bdd-8253-86568fb6874e None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:14.602570 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab843578-679a-4bdd-8253-86568fb6874e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:14.603458 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab843578-679a-4bdd-8253-86568fb6874e None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:14.606481 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 33/63] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:11:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:11:14.616226 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ad5df1-8983-43b5-abd3-ba54b3fa2067 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:14.617263 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ad5df1-8983-43b5-abd3-ba54b3fa2067 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:14.618095 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ad5df1-8983-43b5-abd3-ba54b3fa2067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:14.657394 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-75ad5df1-8983-43b5-abd3-ba54b3fa2067 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:11:14.660290 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-75ad5df1-8983-43b5-abd3-ba54b3fa2067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:11:14.677310 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 31/64] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:11:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:11:14.700236 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:14.723771 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923a714-a2ca-4504-8f84-b629ea214bdc None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:14.820415 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 34/65] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:11:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:11:14.827282 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7569e97-21b7-4de6-bd8b-0f5b170d6f14 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:14.827568 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7569e97-21b7-4de6-bd8b-0f5b170d6f14 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:14.828038 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7569e97-21b7-4de6-bd8b-0f5b170d6f14 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:14.830231 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 35/66] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:11:14 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:11:14.847643 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2131eb4-a976-4b10-89e5-f41539478290 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:14.848204 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2131eb4-a976-4b10-89e5-f41539478290 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:14.848631 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2131eb4-a976-4b10-89e5-f41539478290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:14.876889 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-e2131eb4-a976-4b10-89e5-f41539478290 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:11:14.878691 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e2131eb4-a976-4b10-89e5-f41539478290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:11:14.950336 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:14.982570 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 36/67] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:11:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:11:14.982820 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923a714-a2ca-4504-8f84-b629ea214bdc None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:14.984051 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2131eb4-a976-4b10-89e5-f41539478290 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:15.065835 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c61d486-040c-457c-bd95-ce0454c2c923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:15.067215 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c61d486-040c-457c-bd95-ce0454c2c923 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:15.067885 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c61d486-040c-457c-bd95-ce0454c2c923 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:15.068065 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c61d486-040c-457c-bd95-ce0454c2c923 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:15.088257 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c61d486-040c-457c-bd95-ce0454c2c923 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:11:15.104327 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-3c61d486-040c-457c-bd95-ce0454c2c923 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:11:15.104327 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:11:15.104327 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:11:15.104327 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:11:15.104327 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:11:15.104327 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:11:15.104327 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:11:15.104327 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:11:15.104327 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:11:15.104327 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:11:15.104327 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:11:15.104327 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:11:15.104327 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:11:15.104327 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:11:15.104327 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:11:15.104327 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:11:15.107185 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:11:15.107185 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:11:15.107185 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:11:15.107185 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:11:15.107185 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:11:15.107185 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:11:15.107185 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:11:15.107185 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:11:15.107185 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:11:15.107185 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:11:15.107185 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:11:15.107185 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:11:15.107185 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:11:15.107185 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:11:15.107185 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:11:15.107185 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:11:15.108563 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:11:15.108563 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:11:15.108563 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:11:15.108563 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:11:15.108563 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:11:15.108563 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:11:15.108563 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:11:15.108563 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:11:15.108563 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:11:15.108563 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:11:15.108563 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:11:15.108563 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:11:15.108563 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 32/68] 10.222.0.87 () {56 vars in 1096 bytes} [Tue Jun 21 18:11:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 384 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:11:15.109753 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7831971-11a2-4066-b579-bdb7690e5c4a None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:15.109753 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7831971-11a2-4066-b579-bdb7690e5c4a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:15.111104 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7831971-11a2-4066-b579-bdb7690e5c4a None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:15.119573 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 33/69] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:11:15 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:11:15.121187 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7831971-11a2-4066-b579-bdb7690e5c4a None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:15.125988 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:15.161367 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2131eb4-a976-4b10-89e5-f41539478290 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:15.220566 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de87aaf-d9f8-4284-ba37-0cd4a21b2ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:15.221979 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de87aaf-d9f8-4284-ba37-0cd4a21b2ce8 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:15.222450 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de87aaf-d9f8-4284-ba37-0cd4a21b2ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:15.222948 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de87aaf-d9f8-4284-ba37-0cd4a21b2ce8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:15.242228 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de87aaf-d9f8-4284-ba37-0cd4a21b2ce8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:11:15.251727 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-0de87aaf-d9f8-4284-ba37-0cd4a21b2ce8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:11:15.251727 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:11:15.251727 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:11:15.251727 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:11:15.251727 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:11:15.251727 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:11:15.251727 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:11:15.251727 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:11:15.251727 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:11:15.251727 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:11:15.251727 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:11:15.251727 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:11:15.251727 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:11:15.251727 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:11:15.251727 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:11:15.251727 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:11:15.253413 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:11:15.253413 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:11:15.253413 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:11:15.253413 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:11:15.253413 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:11:15.253413 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:11:15.253413 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:11:15.253413 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:11:15.253413 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:11:15.253413 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:11:15.253413 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:11:15.253413 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:11:15.253413 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:11:15.253413 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:11:15.253413 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:11:15.253413 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:11:15.254977 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:11:15.254977 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:11:15.254977 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:11:15.254977 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:11:15.254977 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:11:15.254977 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:11:15.254977 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:11:15.254977 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:11:15.254977 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:11:15.254977 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:11:15.254977 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:11:15.254977 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:11:15.254977 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 37/70] 10.222.0.87 () {56 vars in 1114 bytes} [Tue Jun 21 18:11:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:11:15.254977 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de87aaf-d9f8-4284-ba37-0cd4a21b2ce8 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:15.304837 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7831971-11a2-4066-b579-bdb7690e5c4a None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:15.360767 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc38d38-1f9e-41f6-8d33-b3a60697799e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:15.362204 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc38d38-1f9e-41f6-8d33-b3a60697799e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:15.362204 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc38d38-1f9e-41f6-8d33-b3a60697799e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:15.362578 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc38d38-1f9e-41f6-8d33-b3a60697799e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:15.379091 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc38d38-1f9e-41f6-8d33-b3a60697799e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:11:15.390870 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-bfc38d38-1f9e-41f6-8d33-b3a60697799e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:11:15.390870 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:11:15.390870 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:11:15.390870 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:11:15.390870 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:11:15.390870 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:11:15.390870 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:11:15.390870 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:11:15.390870 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:11:15.390870 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:11:15.390870 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:11:15.390870 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:11:15.390870 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:11:15.390870 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:11:15.390870 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:11:15.390870 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:11:15.392698 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:11:15.392698 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:11:15.392698 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:11:15.392698 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:11:15.392698 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:11:15.392698 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:11:15.392698 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:11:15.392698 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:11:15.392698 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:11:15.392698 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:11:15.392698 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:11:15.392698 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:11:15.392698 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:11:15.392698 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:11:15.392698 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:11:15.392698 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:11:15.394362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:11:15.394362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:11:15.394362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:11:15.394362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:11:15.394362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:11:15.394362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:11:15.394362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:11:15.394362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:11:15.394362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:11:15.394362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:11:15.394362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:11:15.394362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:11:15.394362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 34/71] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:11:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:11:15.397530 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f171fef-465d-48d7-a944-1c2382ddec71 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:15.397530 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f171fef-465d-48d7-a944-1c2382ddec71 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:15.397530 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f171fef-465d-48d7-a944-1c2382ddec71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:15.407238 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:15.431684 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-4f171fef-465d-48d7-a944-1c2382ddec71 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:11:15.433906 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4f171fef-465d-48d7-a944-1c2382ddec71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:11:15.462699 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de87aaf-d9f8-4284-ba37-0cd4a21b2ce8 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:15.536311 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f50d05-0d42-464e-b3f2-73191d0734d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:15.538438 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f50d05-0d42-464e-b3f2-73191d0734d4 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:15.539015 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f50d05-0d42-464e-b3f2-73191d0734d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:15.539970 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f50d05-0d42-464e-b3f2-73191d0734d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:15.540959 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f50d05-0d42-464e-b3f2-73191d0734d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:15.541683 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f50d05-0d42-464e-b3f2-73191d0734d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:15.544191 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f50d05-0d42-464e-b3f2-73191d0734d4 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:15.566290 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 38/72] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:11:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:15.566290 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f50d05-0d42-464e-b3f2-73191d0734d4 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:15.572979 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 35/73] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:11:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:11:15.574532 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f171fef-465d-48d7-a944-1c2382ddec71 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:15.579864 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:15.717053 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f50d05-0d42-464e-b3f2-73191d0734d4 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:15.727205 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f171fef-465d-48d7-a944-1c2382ddec71 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:15.771489 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc024e8-d2d8-4858-ac0d-e9dfeaa19722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:15.772570 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc024e8-d2d8-4858-ac0d-e9dfeaa19722 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:15.773172 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc024e8-d2d8-4858-ac0d-e9dfeaa19722 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:15.773556 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc024e8-d2d8-4858-ac0d-e9dfeaa19722 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:15.774551 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc024e8-d2d8-4858-ac0d-e9dfeaa19722 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:15.775004 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc024e8-d2d8-4858-ac0d-e9dfeaa19722 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:15.776679 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc024e8-d2d8-4858-ac0d-e9dfeaa19722 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:15.778535 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac2d388-04f4-40a0-a4d4-23ece6ee4b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:15.779364 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2d388-04f4-40a0-a4d4-23ece6ee4b10 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:15.779709 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2d388-04f4-40a0-a4d4-23ece6ee4b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:15.780048 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2d388-04f4-40a0-a4d4-23ece6ee4b10 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:15.780944 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac2d388-04f4-40a0-a4d4-23ece6ee4b10 None admin] RBAC: Adding query filter params (name=member) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:15.781312 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac2d388-04f4-40a0-a4d4-23ece6ee4b10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:15.782901 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac2d388-04f4-40a0-a4d4-23ece6ee4b10 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:15.784976 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:15.796205 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 39/74] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:11:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:15.796205 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad94a7-ad59-4105-9215-b093a3860f4c None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:15.796205 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad94a7-ad59-4105-9215-b093a3860f4c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:15.796205 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad94a7-ad59-4105-9215-b093a3860f4c None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:15.796205 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 40/75] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:11:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:11:15.796205 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad94a7-ad59-4105-9215-b093a3860f4c None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:15.804846 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 36/76] 10.222.0.87 () {56 vars in 1101 bytes} [Tue Jun 21 18:11:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:15.806328 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac2d388-04f4-40a0-a4d4-23ece6ee4b10 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:15.898683 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:15.944314 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad94a7-ad59-4105-9215-b093a3860f4c None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:15.973627 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac2d388-04f4-40a0-a4d4-23ece6ee4b10 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:15.991851 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e6edd9-511d-41e7-a3f6-cc964a65b1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:15.993343 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6edd9-511d-41e7-a3f6-cc964a65b1f0 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:15.993843 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6edd9-511d-41e7-a3f6-cc964a65b1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:15.994548 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6edd9-511d-41e7-a3f6-cc964a65b1f0 None admin] PATH_INFO: `/v3/users/dd94cb8910014827bad3f45b205880f7` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:15.996927 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e6edd9-511d-41e7-a3f6-cc964a65b1f0 None admin] RBAC: Authorizing `identity:get_user(user_id=dd94cb8910014827bad3f45b205880f7)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:15.999480 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e6edd9-511d-41e7-a3f6-cc964a65b1f0 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:16.002367 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 41/77] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:11:15 2022] GET /identity/v3/users/dd94cb8910014827bad3f45b205880f7 => generated 247 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:16.004478 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9da9e-a9b0-4cd5-80b7-a72248865597 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:16.004926 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9da9e-a9b0-4cd5-80b7-a72248865597 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:16.005372 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9da9e-a9b0-4cd5-80b7-a72248865597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:16.027965 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a586df61-d4a7-4e49-90a4-08ec8437938f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:16.028812 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a586df61-d4a7-4e49-90a4-08ec8437938f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:16.029096 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a586df61-d4a7-4e49-90a4-08ec8437938f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:16.029371 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a586df61-d4a7-4e49-90a4-08ec8437938f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:16.032009 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-e8a9da9e-a9b0-4cd5-80b7-a72248865597 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:11:16.033534 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e8a9da9e-a9b0-4cd5-80b7-a72248865597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:11:16.045490 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a586df61-d4a7-4e49-90a4-08ec8437938f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:11:16.056212 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-a586df61-d4a7-4e49-90a4-08ec8437938f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:11:16.056212 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:11:16.056212 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:11:16.056212 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:11:16.056212 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:11:16.056212 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:11:16.056212 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:11:16.056212 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:11:16.056212 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:11:16.056212 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:11:16.056212 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:11:16.056212 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:11:16.056212 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:11:16.056212 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:11:16.056212 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:11:16.056212 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:11:16.058031 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:11:16.058031 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:11:16.058031 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:11:16.058031 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:11:16.058031 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:11:16.058031 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:11:16.058031 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:11:16.058031 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:11:16.058031 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:11:16.058031 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:11:16.058031 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:11:16.058031 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:11:16.058031 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:11:16.058031 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:11:16.058031 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:11:16.058031 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:11:16.060130 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:11:16.060130 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:11:16.060130 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:11:16.060130 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:11:16.060130 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:11:16.060130 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:11:16.060130 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:11:16.060130 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:11:16.060130 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:11:16.060130 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:11:16.060130 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:11:16.060130 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:11:16.060130 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 37/78] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:11:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:11:16.060130 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a586df61-d4a7-4e49-90a4-08ec8437938f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:16.141669 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 42/79] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:11:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:11:16.142874 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:16.143236 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9da9e-a9b0-4cd5-80b7-a72248865597 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:16.240582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a586df61-d4a7-4e49-90a4-08ec8437938f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:16.251459 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:16.314841 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b42e1b2-9be7-4815-853d-9176628cdf3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:16.315960 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42e1b2-9be7-4815-853d-9176628cdf3a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:16.316371 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42e1b2-9be7-4815-853d-9176628cdf3a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:16.316908 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42e1b2-9be7-4815-853d-9176628cdf3a None admin] PATH_INFO: `/v3/users/786bcdb4dc274deb8280030ff16c6d0a` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:16.323663 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9da9e-a9b0-4cd5-80b7-a72248865597 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:16.344034 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b42e1b2-9be7-4815-853d-9176628cdf3a None admin] RBAC: Authorizing `identity:get_user(user_id=786bcdb4dc274deb8280030ff16c6d0a)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:16.347248 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b42e1b2-9be7-4815-853d-9176628cdf3a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:16.351432 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 38/80] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:11:16 2022] GET /identity/v3/users/786bcdb4dc274deb8280030ff16c6d0a => generated 275 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:16.353002 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b42e1b2-9be7-4815-853d-9176628cdf3a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:16.385240 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d6975e-108d-4e78-b766-ed9cddf3ecff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:16.387218 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6975e-108d-4e78-b766-ed9cddf3ecff None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:16.387731 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6975e-108d-4e78-b766-ed9cddf3ecff None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:16.388258 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6975e-108d-4e78-b766-ed9cddf3ecff None admin] PATH_INFO: `/v3/users/786bcdb4dc274deb8280030ff16c6d0a` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:16.390661 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d6975e-108d-4e78-b766-ed9cddf3ecff None admin] RBAC: Authorizing `identity:get_user(user_id=786bcdb4dc274deb8280030ff16c6d0a)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:16.393488 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d6975e-108d-4e78-b766-ed9cddf3ecff None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:16.397250 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 43/81] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:11:16 2022] GET /identity/v3/users/786bcdb4dc274deb8280030ff16c6d0a => generated 275 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:16.398684 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d6975e-108d-4e78-b766-ed9cddf3ecff None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:16.561613 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b42e1b2-9be7-4815-853d-9176628cdf3a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:16.589072 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:16.603285 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d6975e-108d-4e78-b766-ed9cddf3ecff None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:16.633383 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2045775-4411-45b2-8d2c-0437901d5b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:16.634216 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2045775-4411-45b2-8d2c-0437901d5b33 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:16.634517 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2045775-4411-45b2-8d2c-0437901d5b33 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:16.634798 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2045775-4411-45b2-8d2c-0437901d5b33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:16.659200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30664605-f73e-4e22-9b14-77343dead2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:16.659904 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30664605-f73e-4e22-9b14-77343dead2e7 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:16.660192 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30664605-f73e-4e22-9b14-77343dead2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:16.660485 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30664605-f73e-4e22-9b14-77343dead2e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:16.661284 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30664605-f73e-4e22-9b14-77343dead2e7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:16.661600 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30664605-f73e-4e22-9b14-77343dead2e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:16.663257 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30664605-f73e-4e22-9b14-77343dead2e7 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:16.667294 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:16.682279 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 44/82] 10.222.0.87 () {56 vars in 1101 bytes} [Tue Jun 21 18:11:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:16.684577 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30664605-f73e-4e22-9b14-77343dead2e7 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:16.735286 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2045775-4411-45b2-8d2c-0437901d5b33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:16.737224 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2045775-4411-45b2-8d2c-0437901d5b33 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:16.865488 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30664605-f73e-4e22-9b14-77343dead2e7 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:16.919696 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd0f20-53c7-447a-9555-0a906019ad99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:16.921033 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd0f20-53c7-447a-9555-0a906019ad99 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:16.921482 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd0f20-53c7-447a-9555-0a906019ad99 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:16.921973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd0f20-53c7-447a-9555-0a906019ad99 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:16.922622 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 39/83] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:11:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:11:16.924030 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2045775-4411-45b2-8d2c-0437901d5b33 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:16.924214 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bd0f20-53c7-447a-9555-0a906019ad99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:16.926109 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bd0f20-53c7-447a-9555-0a906019ad99 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:16.929147 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 45/84] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:11:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:16.931504 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd0f20-53c7-447a-9555-0a906019ad99 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:16.958825 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:17.044168 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:17.101708 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2045775-4411-45b2-8d2c-0437901d5b33 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:17.105465 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd0f20-53c7-447a-9555-0a906019ad99 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:17.161734 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7575972f-e088-4f57-8c08-f7bd1fb6f58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:17.162588 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575972f-e088-4f57-8c08-f7bd1fb6f58f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:17.162952 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575972f-e088-4f57-8c08-f7bd1fb6f58f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:17.163298 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575972f-e088-4f57-8c08-f7bd1fb6f58f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:17.163590 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b2874f-c494-4cb5-a6a3-a42aff4e76a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:17.164654 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b2874f-c494-4cb5-a6a3-a42aff4e76a6 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:17.165088 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b2874f-c494-4cb5-a6a3-a42aff4e76a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:17.165534 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b2874f-c494-4cb5-a6a3-a42aff4e76a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:17.261076 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7575972f-e088-4f57-8c08-f7bd1fb6f58f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:17.262949 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7575972f-e088-4f57-8c08-f7bd1fb6f58f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:17.288166 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:17.305554 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b2874f-c494-4cb5-a6a3-a42aff4e76a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:17.307587 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b2874f-c494-4cb5-a6a3-a42aff4e76a6 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:17.400381 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:17.409542 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 40/85] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:11:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:11:17.410486 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7575972f-e088-4f57-8c08-f7bd1fb6f58f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:17.469723 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 46/86] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:11:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:11:17.470764 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b2874f-c494-4cb5-a6a3-a42aff4e76a6 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:17.569143 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7575972f-e088-4f57-8c08-f7bd1fb6f58f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:17.598192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:17.641724 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd04c04-10a9-43af-b138-4b5c0a84925e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:17.643371 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd04c04-10a9-43af-b138-4b5c0a84925e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:17.643885 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd04c04-10a9-43af-b138-4b5c0a84925e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:17.644389 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd04c04-10a9-43af-b138-4b5c0a84925e None admin] PATH_INFO: `/v3/users/786bcdb4dc274deb8280030ff16c6d0a` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:17.646915 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd04c04-10a9-43af-b138-4b5c0a84925e None admin] RBAC: Authorizing `identity:get_user(user_id=786bcdb4dc274deb8280030ff16c6d0a)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:17.649622 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd04c04-10a9-43af-b138-4b5c0a84925e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:17.653252 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 41/87] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:11:17 2022] GET /identity/v3/users/786bcdb4dc274deb8280030ff16c6d0a => generated 275 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:17.654537 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd04c04-10a9-43af-b138-4b5c0a84925e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:17.664863 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b2874f-c494-4cb5-a6a3-a42aff4e76a6 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:17.718059 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe855d21-cba9-435a-a7c5-14eab33635be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:17.719099 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe855d21-cba9-435a-a7c5-14eab33635be None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:17.719381 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe855d21-cba9-435a-a7c5-14eab33635be None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:17.719657 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe855d21-cba9-435a-a7c5-14eab33635be None admin] PATH_INFO: `/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:17.720980 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe855d21-cba9-435a-a7c5-14eab33635be None admin] RBAC: Authorizing `identity:get_project(project_id=48caa0e9d7fe40d28e99abd1a7a6896f)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:17.722690 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe855d21-cba9-435a-a7c5-14eab33635be None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:17.725347 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 47/88] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:11:17 2022] GET /identity/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:17.726688 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe855d21-cba9-435a-a7c5-14eab33635be None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:17.779266 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:17.867042 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd04c04-10a9-43af-b138-4b5c0a84925e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:17.919117 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe855d21-cba9-435a-a7c5-14eab33635be None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:17.942659 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee31b95b-fc2f-44f9-b625-31e99c34e8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:17.943875 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31b95b-fc2f-44f9-b625-31e99c34e8cc None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:17.944345 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31b95b-fc2f-44f9-b625-31e99c34e8cc None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:17.944789 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31b95b-fc2f-44f9-b625-31e99c34e8cc None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:17.946102 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee31b95b-fc2f-44f9-b625-31e99c34e8cc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:17.948967 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee31b95b-fc2f-44f9-b625-31e99c34e8cc None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:17.969321 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5919fe-ec21-4114-adee-46e4ed9baa00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:17.970647 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5919fe-ec21-4114-adee-46e4ed9baa00 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:17.971027 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5919fe-ec21-4114-adee-46e4ed9baa00 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:17.971422 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5919fe-ec21-4114-adee-46e4ed9baa00 None admin] PATH_INFO: `/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:17.972948 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5919fe-ec21-4114-adee-46e4ed9baa00 None admin] RBAC: Authorizing `identity:get_project(project_id=48caa0e9d7fe40d28e99abd1a7a6896f)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:17.976042 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5919fe-ec21-4114-adee-46e4ed9baa00 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:17.979207 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 48/89] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:11:17 2022] GET /identity/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f => generated 298 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:17.979207 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5919fe-ec21-4114-adee-46e4ed9baa00 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:17.989591 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 42/90] 10.222.0.87 () {60 vars in 1132 bytes} [Tue Jun 21 18:11:17 2022] POST /identity/v3/users => generated 289 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:11:17.991391 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee31b95b-fc2f-44f9-b625-31e99c34e8cc None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:18.077508 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:18.198602 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee31b95b-fc2f-44f9-b625-31e99c34e8cc None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:18.211833 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:18.218373 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5919fe-ec21-4114-adee-46e4ed9baa00 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:18.286015 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16b97c-49b5-4a41-92fb-b497ed9dcfb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:18.288119 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16b97c-49b5-4a41-92fb-b497ed9dcfb9 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:18.289946 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16b97c-49b5-4a41-92fb-b497ed9dcfb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:18.289946 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16b97c-49b5-4a41-92fb-b497ed9dcfb9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:18.297501 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc907c4-99b6-46bc-b079-be2ddbf8ef69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:18.298697 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc907c4-99b6-46bc-b079-be2ddbf8ef69 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:18.299171 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc907c4-99b6-46bc-b079-be2ddbf8ef69 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:18.299574 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc907c4-99b6-46bc-b079-be2ddbf8ef69 None admin] PATH_INFO: `/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:18.302255 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc907c4-99b6-46bc-b079-be2ddbf8ef69 None admin] RBAC: Authorizing `identity:get_project(project_id=48caa0e9d7fe40d28e99abd1a7a6896f)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:18.305889 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc907c4-99b6-46bc-b079-be2ddbf8ef69 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:18.309349 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 49/91] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:11:17 2022] GET /identity/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f => generated 298 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:18.311863 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc907c4-99b6-46bc-b079-be2ddbf8ef69 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:18.463685 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be16b97c-49b5-4a41-92fb-b497ed9dcfb9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:18.466406 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be16b97c-49b5-4a41-92fb-b497ed9dcfb9 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:18.562085 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc907c4-99b6-46bc-b079-be2ddbf8ef69 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:18.571851 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:18.644040 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d23b1-d45b-4d29-922c-7b979e2da913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:18.645159 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d23b1-d45b-4d29-922c-7b979e2da913 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:18.645663 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d23b1-d45b-4d29-922c-7b979e2da913 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:18.646448 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d23b1-d45b-4d29-922c-7b979e2da913 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:18.647811 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d23b1-d45b-4d29-922c-7b979e2da913 None admin] RBAC: Adding query filter params (role.id=98f0943a610f4a9e9f8de54f6e9f717d, scope.project.id=48caa0e9d7fe40d28e99abd1a7a6896f, user.id=dd94cb8910014827bad3f45b205880f7) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:18.648363 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d23b1-d45b-4d29-922c-7b979e2da913 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:18.651239 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d23b1-d45b-4d29-922c-7b979e2da913 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:18.663525 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 50/92] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:11:18 2022] GET /identity/v3/role_assignments?user.id=dd94cb8910014827bad3f45b205880f7&scope.project.id=48caa0e9d7fe40d28e99abd1a7a6896f&role.id=98f0943a610f4a9e9f8de54f6e9f717d => generated 259 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:18.665212 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d23b1-d45b-4d29-922c-7b979e2da913 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:18.697694 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 43/93] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:11:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 707 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:11:18.700989 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16b97c-49b5-4a41-92fb-b497ed9dcfb9 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:18.769499 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:18.993749 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16b97c-49b5-4a41-92fb-b497ed9dcfb9 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:19.012967 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d23b1-d45b-4d29-922c-7b979e2da913 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:19.063161 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a9d3b-2e84-48f9-b136-83ef49fbe945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:19.064057 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a9d3b-2e84-48f9-b136-83ef49fbe945 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:19.064357 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a9d3b-2e84-48f9-b136-83ef49fbe945 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:19.064643 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a9d3b-2e84-48f9-b136-83ef49fbe945 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:19.065512 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a9d3b-2e84-48f9-b136-83ef49fbe945 None admin] RBAC: Adding query filter params (role.id=da6d4a0ab8c04809ba120170fb4ae8a9, scope.project.id=48caa0e9d7fe40d28e99abd1a7a6896f, user.id=786bcdb4dc274deb8280030ff16c6d0a) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:19.067188 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a9d3b-2e84-48f9-b136-83ef49fbe945 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:19.069371 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a9d3b-2e84-48f9-b136-83ef49fbe945 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:19.076061 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 44/94] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:11:18 2022] GET /identity/v3/role_assignments?user.id=786bcdb4dc274deb8280030ff16c6d0a&scope.project.id=48caa0e9d7fe40d28e99abd1a7a6896f&role.id=da6d4a0ab8c04809ba120170fb4ae8a9 => generated 259 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:19.076944 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a9d3b-2e84-48f9-b136-83ef49fbe945 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:19.081053 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:19.088788 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f83c28-8faf-40c7-a413-0648ebec103c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:19.090002 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f83c28-8faf-40c7-a413-0648ebec103c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:19.090504 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f83c28-8faf-40c7-a413-0648ebec103c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:19.091160 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f83c28-8faf-40c7-a413-0648ebec103c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:19.092824 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f83c28-8faf-40c7-a413-0648ebec103c None admin] RBAC: Adding query filter params (role.id=46e77bb17fc3465496a7a3a29d210042, scope.project.id=48caa0e9d7fe40d28e99abd1a7a6896f, user.id=786bcdb4dc274deb8280030ff16c6d0a) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:19.093367 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f83c28-8faf-40c7-a413-0648ebec103c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:19.096527 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f83c28-8faf-40c7-a413-0648ebec103c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:19.104928 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 51/95] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:11:18 2022] GET /identity/v3/role_assignments?user.id=786bcdb4dc274deb8280030ff16c6d0a&scope.project.id=48caa0e9d7fe40d28e99abd1a7a6896f&role.id=46e77bb17fc3465496a7a3a29d210042 => generated 259 bytes in 441 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:19.317734 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a9d3b-2e84-48f9-b136-83ef49fbe945 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:19.390245 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f5c34e-6040-4ce1-90e7-ec02136bc781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:19.391833 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5c34e-6040-4ce1-90e7-ec02136bc781 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:19.392346 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5c34e-6040-4ce1-90e7-ec02136bc781 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:19.392844 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5c34e-6040-4ce1-90e7-ec02136bc781 None admin] PATH_INFO: `/v3/projects/9abd12a809ab4ddfb059927f8acd84af` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:19.395257 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f5c34e-6040-4ce1-90e7-ec02136bc781 None admin] RBAC: Authorizing `identity:get_project(project_id=9abd12a809ab4ddfb059927f8acd84af)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:19.398261 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f5c34e-6040-4ce1-90e7-ec02136bc781 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:19.402087 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 45/96] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:11:19 2022] GET /identity/v3/projects/9abd12a809ab4ddfb059927f8acd84af => generated 312 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:19.411696 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f83c28-8faf-40c7-a413-0648ebec103c None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:19.627318 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:19.639350 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f83c28-8faf-40c7-a413-0648ebec103c None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:19.711991 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb42fb-b7e3-4071-8dd4-97dc451d11fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:19.713625 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb42fb-b7e3-4071-8dd4-97dc451d11fa None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:19.713964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb42fb-b7e3-4071-8dd4-97dc451d11fa None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:19.714676 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb42fb-b7e3-4071-8dd4-97dc451d11fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:19.716071 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffb42fb-b7e3-4071-8dd4-97dc451d11fa None admin] RBAC: Adding query filter params (role.id=da6d4a0ab8c04809ba120170fb4ae8a9, scope.project.id=9abd12a809ab4ddfb059927f8acd84af, user.id=786bcdb4dc274deb8280030ff16c6d0a) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:19.716633 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffb42fb-b7e3-4071-8dd4-97dc451d11fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:19.719524 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffb42fb-b7e3-4071-8dd4-97dc451d11fa None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:19.733606 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 52/97] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:11:19 2022] GET /identity/v3/role_assignments?user.id=786bcdb4dc274deb8280030ff16c6d0a&scope.project.id=9abd12a809ab4ddfb059927f8acd84af&role.id=da6d4a0ab8c04809ba120170fb4ae8a9 => generated 259 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:21.961091 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c627e7fc-6d4e-464a-a8a1-66b18046d3a0 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:21.962191 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c627e7fc-6d4e-464a-a8a1-66b18046d3a0 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:21.962688 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c627e7fc-6d4e-464a-a8a1-66b18046d3a0 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:21.964386 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 46/98] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:11:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:11:21.970716 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52104805-bd52-4421-9285-5e4e5968bedb None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:21.971715 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52104805-bd52-4421-9285-5e4e5968bedb None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:21.972213 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52104805-bd52-4421-9285-5e4e5968bedb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:22.003334 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-52104805-bd52-4421-9285-5e4e5968bedb None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:11:22.005108 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-52104805-bd52-4421-9285-5e4e5968bedb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:11:22.102404 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 53/99] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:11:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:11:22.133977 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c627e7fc-6d4e-464a-a8a1-66b18046d3a0 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:22.250200 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:22.279893 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c627e7fc-6d4e-464a-a8a1-66b18046d3a0 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:22.321710 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef747a2e-db29-4b2d-845c-166745592a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:22.322584 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef747a2e-db29-4b2d-845c-166745592a7d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:22.322875 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef747a2e-db29-4b2d-845c-166745592a7d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:22.323161 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef747a2e-db29-4b2d-845c-166745592a7d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:22.338173 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef747a2e-db29-4b2d-845c-166745592a7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:11:22.346221 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-ef747a2e-db29-4b2d-845c-166745592a7d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:11:22.346221 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:11:22.346221 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:11:22.346221 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:11:22.346221 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:11:22.346221 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:11:22.346221 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:11:22.346221 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:11:22.346221 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:11:22.346221 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:11:22.346221 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:11:22.346221 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:11:22.346221 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:11:22.346221 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:11:22.346221 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:11:22.346221 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:11:22.347157 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:11:22.347157 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:11:22.347157 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:11:22.347157 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:11:22.347157 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:11:22.347157 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:11:22.347157 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:11:22.347157 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:11:22.347157 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:11:22.347157 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:11:22.347157 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:11:22.347157 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:11:22.347157 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:11:22.347157 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:11:22.347157 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:11:22.347157 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:11:22.348000 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:11:22.348000 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:11:22.348000 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:11:22.348000 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:11:22.348000 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:11:22.348000 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:11:22.348000 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:11:22.348000 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:11:22.348000 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:11:22.348000 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:11:22.348000 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:11:22.348000 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:11:22.348587 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 47/100] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:11:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:11:22.355696 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52104805-bd52-4421-9285-5e4e5968bedb None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:22.470702 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:22.483173 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52104805-bd52-4421-9285-5e4e5968bedb None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:22.535859 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa200d-d75d-4bdf-9531-c73bd51e0714 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:22.536421 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa200d-d75d-4bdf-9531-c73bd51e0714 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:22.537001 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa200d-d75d-4bdf-9531-c73bd51e0714 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:22.540326 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 48/101] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:11:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:11:22.547940 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555415a1-5e2d-4a81-b018-7f7a469436d6 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:22.548458 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555415a1-5e2d-4a81-b018-7f7a469436d6 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:22.548958 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555415a1-5e2d-4a81-b018-7f7a469436d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:22.583048 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca897ca-2037-4bce-b659-d0cb300ce787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:22.584313 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca897ca-2037-4bce-b659-d0cb300ce787 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:22.584830 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca897ca-2037-4bce-b659-d0cb300ce787 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:22.585330 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca897ca-2037-4bce-b659-d0cb300ce787 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:22.586824 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca897ca-2037-4bce-b659-d0cb300ce787 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:22.587379 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca897ca-2037-4bce-b659-d0cb300ce787 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:22.590070 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca897ca-2037-4bce-b659-d0cb300ce787 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:22.608620 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-555415a1-5e2d-4a81-b018-7f7a469436d6 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:11:22.611265 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-555415a1-5e2d-4a81-b018-7f7a469436d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:11:22.625156 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 54/102] 10.222.0.87 () {56 vars in 1101 bytes} [Tue Jun 21 18:11:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:22.633293 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca897ca-2037-4bce-b659-d0cb300ce787 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:22.922940 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 49/103] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:11:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:11:22.924848 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9153a53c-cb23-46d1-9fb3-4a70cfd9fce1 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:22.925393 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9153a53c-cb23-46d1-9fb3-4a70cfd9fce1 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:22.925949 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9153a53c-cb23-46d1-9fb3-4a70cfd9fce1 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:22.928328 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 50/104] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:11:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:11:22.942015 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05041b4c-7c1c-4872-9fea-6b6f06a93018 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:22.942654 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05041b4c-7c1c-4872-9fea-6b6f06a93018 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:22.942809 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca897ca-2037-4bce-b659-d0cb300ce787 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:22.943405 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05041b4c-7c1c-4872-9fea-6b6f06a93018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:22.972014 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-05041b4c-7c1c-4872-9fea-6b6f06a93018 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:11:22.974184 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-05041b4c-7c1c-4872-9fea-6b6f06a93018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:11:23.027861 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0771d15-f001-445c-8cda-4c05e27696b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:23.029403 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0771d15-f001-445c-8cda-4c05e27696b2 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:23.029930 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0771d15-f001-445c-8cda-4c05e27696b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:23.030422 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0771d15-f001-445c-8cda-4c05e27696b2 None admin] PATH_INFO: `/v3/users/2ed789dda2154c888945486c0b8c55d3` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:23.095543 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0771d15-f001-445c-8cda-4c05e27696b2 None admin] RBAC: Authorizing `identity:get_user(user_id=2ed789dda2154c888945486c0b8c55d3)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:23.097180 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0771d15-f001-445c-8cda-4c05e27696b2 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:23.100278 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 55/105] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:11:22 2022] GET /identity/v3/users/2ed789dda2154c888945486c0b8c55d3 => generated 289 bytes in 469 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:11:23.102184 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0771d15-f001-445c-8cda-4c05e27696b2 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:23.126641 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:23.142243 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:23.182825 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 51/106] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:11:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:11:23.185622 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05041b4c-7c1c-4872-9fea-6b6f06a93018 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:23.360310 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0771d15-f001-445c-8cda-4c05e27696b2 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:23.371090 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05041b4c-7c1c-4872-9fea-6b6f06a93018 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:23.416712 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a05e38-d614-471b-a0b5-39285784a41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:23.417804 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a05e38-d614-471b-a0b5-39285784a41f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:23.418314 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a05e38-d614-471b-a0b5-39285784a41f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:23.418790 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a05e38-d614-471b-a0b5-39285784a41f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:23.433221 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaa2064-a311-4100-a902-b6d78c94ed76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:23.434652 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa2064-a311-4100-a902-b6d78c94ed76 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:23.435406 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa2064-a311-4100-a902-b6d78c94ed76 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:23.436129 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa2064-a311-4100-a902-b6d78c94ed76 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:23.438849 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaa2064-a311-4100-a902-b6d78c94ed76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:23.442069 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaa2064-a311-4100-a902-b6d78c94ed76 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:23.446202 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 56/107] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:11:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:23.449051 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a8e7d-880a-4d65-b24a-3b81f011974d None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:23.450169 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a8e7d-880a-4d65-b24a-3b81f011974d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:23.451025 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a8e7d-880a-4d65-b24a-3b81f011974d None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:23.454113 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 57/108] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:11:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:11:23.455983 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a8e7d-880a-4d65-b24a-3b81f011974d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:23.517272 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:23.546461 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a05e38-d614-471b-a0b5-39285784a41f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:23.549976 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a05e38-d614-471b-a0b5-39285784a41f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:23.631128 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:23.678194 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a8e7d-880a-4d65-b24a-3b81f011974d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:23.746181 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 52/109] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:11:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:11:23.747397 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a05e38-d614-471b-a0b5-39285784a41f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:23.749078 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68c1e08-8cd1-4138-9966-31dcd3b84c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:23.750816 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c1e08-8cd1-4138-9966-31dcd3b84c73 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:23.751640 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c1e08-8cd1-4138-9966-31dcd3b84c73 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:23.752408 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c1e08-8cd1-4138-9966-31dcd3b84c73 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:23.774360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68c1e08-8cd1-4138-9966-31dcd3b84c73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:11:23.791090 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-e68c1e08-8cd1-4138-9966-31dcd3b84c73 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:11:23.791090 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:11:23.791090 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:11:23.791090 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:11:23.791090 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:11:23.791090 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:11:23.791090 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:11:23.791090 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:11:23.791090 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:11:23.791090 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:11:23.791090 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:11:23.791090 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:11:23.791090 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:11:23.791090 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:11:23.791090 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:11:23.791090 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:11:23.792766 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:11:23.792766 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:11:23.792766 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:11:23.792766 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:11:23.792766 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:11:23.792766 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:11:23.792766 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:11:23.792766 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:11:23.792766 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:11:23.792766 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:11:23.792766 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:11:23.792766 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:11:23.792766 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:11:23.792766 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:11:23.792766 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:11:23.792766 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:11:23.794311 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:11:23.794311 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:11:23.794311 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:11:23.794311 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:11:23.794311 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:11:23.794311 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:11:23.794311 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:11:23.794311 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:11:23.794311 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:11:23.794311 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:11:23.794311 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:11:23.794311 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:11:23.806045 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 58/110] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:11:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:11:23.820132 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae0fd6-361e-4ecf-9bdb-8d77d6babd46 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:23.820937 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae0fd6-361e-4ecf-9bdb-8d77d6babd46 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:23.821744 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae0fd6-361e-4ecf-9bdb-8d77d6babd46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:23.863803 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-a3ae0fd6-361e-4ecf-9bdb-8d77d6babd46 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:11:23.866821 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a3ae0fd6-361e-4ecf-9bdb-8d77d6babd46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:11:23.868578 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:23.883478 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a05e38-d614-471b-a0b5-39285784a41f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:23.926886 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e2194-4428-417f-a06d-5945613e796b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:23.927631 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e2194-4428-417f-a06d-5945613e796b None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:23.927925 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e2194-4428-417f-a06d-5945613e796b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:23.928211 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e2194-4428-417f-a06d-5945613e796b None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:23.929057 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538e2194-4428-417f-a06d-5945613e796b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:23.931116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538e2194-4428-417f-a06d-5945613e796b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:23.968885 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 53/111] 10.222.0.87 () {60 vars in 1141 bytes} [Tue Jun 21 18:11:23 2022] POST /identity/v3/projects => generated 302 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:11:23.970235 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e2194-4428-417f-a06d-5945613e796b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:24.049743 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 59/112] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:11:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:11:24.059544 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae0fd6-361e-4ecf-9bdb-8d77d6babd46 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:24.201212 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:24.208580 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e2194-4428-417f-a06d-5945613e796b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:24.261360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae0fd6-361e-4ecf-9bdb-8d77d6babd46 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:24.283503 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c99452d-a89c-4d1b-81ef-30a27bc834e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:24.284774 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99452d-a89c-4d1b-81ef-30a27bc834e6 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:24.285109 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99452d-a89c-4d1b-81ef-30a27bc834e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:24.285540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99452d-a89c-4d1b-81ef-30a27bc834e6 None admin] PATH_INFO: `/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:24.287426 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c99452d-a89c-4d1b-81ef-30a27bc834e6 None admin] RBAC: Authorizing `identity:get_project(project_id=48caa0e9d7fe40d28e99abd1a7a6896f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:24.290237 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c99452d-a89c-4d1b-81ef-30a27bc834e6 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:24.292773 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 54/113] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:11:23 2022] GET /identity/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f => generated 298 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:24.295022 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c99452d-a89c-4d1b-81ef-30a27bc834e6 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:24.318772 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:24.336893 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f237adff-c228-41c0-b8c7-cc2e4b90f459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:24.338283 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f237adff-c228-41c0-b8c7-cc2e4b90f459 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:24.338872 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f237adff-c228-41c0-b8c7-cc2e4b90f459 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:24.339428 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f237adff-c228-41c0-b8c7-cc2e4b90f459 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:24.340944 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f237adff-c228-41c0-b8c7-cc2e4b90f459 None admin] RBAC: Adding query filter params (name=member) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:24.341491 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f237adff-c228-41c0-b8c7-cc2e4b90f459 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:24.344352 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f237adff-c228-41c0-b8c7-cc2e4b90f459 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:24.362742 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 60/114] 10.222.0.87 () {56 vars in 1101 bytes} [Tue Jun 21 18:11:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:24.366185 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3c03e-24a6-4d45-8039-c41bd5814c3d None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:24.366717 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3c03e-24a6-4d45-8039-c41bd5814c3d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:24.367262 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3c03e-24a6-4d45-8039-c41bd5814c3d None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:24.370085 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 61/115] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:11:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:11:24.378144 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd3c03e-24a6-4d45-8039-c41bd5814c3d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:24.431867 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c99452d-a89c-4d1b-81ef-30a27bc834e6 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:24.477812 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd896aa-be2b-47fb-b188-726f6f8e25df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:24.480508 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd896aa-be2b-47fb-b188-726f6f8e25df None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:24.480508 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd896aa-be2b-47fb-b188-726f6f8e25df None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:24.480508 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd896aa-be2b-47fb-b188-726f6f8e25df None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:24.493826 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd896aa-be2b-47fb-b188-726f6f8e25df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:11:24.501842 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-fdd896aa-be2b-47fb-b188-726f6f8e25df None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:11:24.501842 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:11:24.501842 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:11:24.501842 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:11:24.501842 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:11:24.501842 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:11:24.501842 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:11:24.501842 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:11:24.501842 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:11:24.501842 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:11:24.501842 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:11:24.501842 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:11:24.501842 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:11:24.501842 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:11:24.501842 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:11:24.501842 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:11:24.505028 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:11:24.505028 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:11:24.505028 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:11:24.505028 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:11:24.505028 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:11:24.505028 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:11:24.505028 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:11:24.505028 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:11:24.505028 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:11:24.505028 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:11:24.505028 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:11:24.505028 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:11:24.505028 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:11:24.505028 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:11:24.505028 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:11:24.505028 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:11:24.508601 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:11:24.508601 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:11:24.508601 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:11:24.508601 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:11:24.508601 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:11:24.508601 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:11:24.508601 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:11:24.508601 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:11:24.508601 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:11:24.508601 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:11:24.508601 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:11:24.508601 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:11:24.508601 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 55/116] 10.222.0.87 () {56 vars in 1096 bytes} [Tue Jun 21 18:11:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:11:24.508601 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd896aa-be2b-47fb-b188-726f6f8e25df None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:24.548457 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd3c03e-24a6-4d45-8039-c41bd5814c3d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:24.582650 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:24.591819 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc5efda-218e-4a68-bb8a-5a30d67877d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:24.592679 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5efda-218e-4a68-bb8a-5a30d67877d4 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:24.593150 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:24.594171 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5efda-218e-4a68-bb8a-5a30d67877d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:24.594515 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5efda-218e-4a68-bb8a-5a30d67877d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:24.595607 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc5efda-218e-4a68-bb8a-5a30d67877d4 None admin] RBAC: Adding query filter params (role.id=5efb4931668c4edf87773a1f5d47f005, scope.project.id=48caa0e9d7fe40d28e99abd1a7a6896f, user.id=2ed789dda2154c888945486c0b8c55d3) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:24.595888 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc5efda-218e-4a68-bb8a-5a30d67877d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:24.597428 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc5efda-218e-4a68-bb8a-5a30d67877d4 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:24.605600 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 62/117] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:11:24 2022] GET /identity/v3/role_assignments?user.id=2ed789dda2154c888945486c0b8c55d3&scope.project.id=48caa0e9d7fe40d28e99abd1a7a6896f&role.id=5efb4931668c4edf87773a1f5d47f005 => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:24.606855 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846276e6-03b0-42e9-8586-ea913be48b8e None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:24.607159 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846276e6-03b0-42e9-8586-ea913be48b8e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:24.607449 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846276e6-03b0-42e9-8586-ea913be48b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:24.636145 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-846276e6-03b0-42e9-8586-ea913be48b8e None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:11:24.639028 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-846276e6-03b0-42e9-8586-ea913be48b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:11:24.641600 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd896aa-be2b-47fb-b188-726f6f8e25df None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:24.689351 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e72e867-4243-4182-bde3-1d923ac964ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:24.690534 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72e867-4243-4182-bde3-1d923ac964ee None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:24.691009 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72e867-4243-4182-bde3-1d923ac964ee None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:24.691627 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72e867-4243-4182-bde3-1d923ac964ee None admin] PATH_INFO: `/v3/users/786bcdb4dc274deb8280030ff16c6d0a` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:24.693737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e72e867-4243-4182-bde3-1d923ac964ee None admin] RBAC: Authorizing `identity:get_user(user_id=786bcdb4dc274deb8280030ff16c6d0a)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:24.695544 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e72e867-4243-4182-bde3-1d923ac964ee None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:24.702339 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 56/118] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:11:24 2022] GET /identity/v3/users/786bcdb4dc274deb8280030ff16c6d0a => generated 275 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:24.702339 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e72e867-4243-4182-bde3-1d923ac964ee None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:24.754003 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 63/119] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:11:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:11:24.755968 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00804559-dff7-4464-b631-8a8dd118f69d None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:24.756519 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00804559-dff7-4464-b631-8a8dd118f69d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:24.757000 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00804559-dff7-4464-b631-8a8dd118f69d None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:24.764218 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 64/120] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:11:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:11:24.765725 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00804559-dff7-4464-b631-8a8dd118f69d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:24.916482 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00804559-dff7-4464-b631-8a8dd118f69d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:24.924110 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e72e867-4243-4182-bde3-1d923ac964ee None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:24.966390 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e98b5b8-9f4d-468e-a1d2-4515d7a4d3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:24.967566 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98b5b8-9f4d-468e-a1d2-4515d7a4d3de None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:24.968066 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98b5b8-9f4d-468e-a1d2-4515d7a4d3de None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:24.968235 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:24.969182 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98b5b8-9f4d-468e-a1d2-4515d7a4d3de None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:24.970548 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e98b5b8-9f4d-468e-a1d2-4515d7a4d3de None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:24.971077 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e98b5b8-9f4d-468e-a1d2-4515d7a4d3de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:24.972877 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e98b5b8-9f4d-468e-a1d2-4515d7a4d3de None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:24.980999 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:24.985961 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 57/121] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:11:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:24.987707 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25bcaaf-22cb-49f0-985b-b3cfe40cf029 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:24.988126 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25bcaaf-22cb-49f0-985b-b3cfe40cf029 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:24.988538 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25bcaaf-22cb-49f0-985b-b3cfe40cf029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:24.995341 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa67e1-530c-479c-91dd-5f2f1bc547fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:24.997538 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa67e1-530c-479c-91dd-5f2f1bc547fa None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:24.998351 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa67e1-530c-479c-91dd-5f2f1bc547fa None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:24.999031 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa67e1-530c-479c-91dd-5f2f1bc547fa None admin] PATH_INFO: `/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:25.001782 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aa67e1-530c-479c-91dd-5f2f1bc547fa None admin] RBAC: Authorizing `identity:get_project(project_id=48caa0e9d7fe40d28e99abd1a7a6896f)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:25.005119 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aa67e1-530c-479c-91dd-5f2f1bc547fa None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:25.009386 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 65/122] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:11:24 2022] GET /identity/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f => generated 298 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:25.010762 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d25bcaaf-22cb-49f0-985b-b3cfe40cf029 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:11:25.011504 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa67e1-530c-479c-91dd-5f2f1bc547fa None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:25.012249 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d25bcaaf-22cb-49f0-985b-b3cfe40cf029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:11:25.100271 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 58/123] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:11:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:11:25.101154 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25bcaaf-22cb-49f0-985b-b3cfe40cf029 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:25.213763 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa67e1-530c-479c-91dd-5f2f1bc547fa None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:25.239767 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25bcaaf-22cb-49f0-985b-b3cfe40cf029 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:25.259573 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e0829-ce8e-4916-be07-ca368e3abcac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:25.260429 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e0829-ce8e-4916-be07-ca368e3abcac None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:25.260719 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e0829-ce8e-4916-be07-ca368e3abcac None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:25.261003 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e0829-ce8e-4916-be07-ca368e3abcac None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:25.274589 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773e0829-ce8e-4916-be07-ca368e3abcac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:11:25.283271 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:25.284744 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-773e0829-ce8e-4916-be07-ca368e3abcac None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:11:25.284744 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:11:25.284744 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:11:25.284744 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:11:25.284744 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:11:25.284744 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:11:25.284744 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:11:25.284744 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:11:25.284744 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:11:25.284744 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:11:25.284744 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:11:25.284744 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:11:25.284744 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:11:25.284744 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:11:25.284744 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:11:25.284744 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:11:25.285586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:11:25.285586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:11:25.285586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:11:25.285586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:11:25.285586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:11:25.285586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:11:25.285586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:11:25.285586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:11:25.285586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:11:25.285586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:11:25.285586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:11:25.285586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:11:25.285586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:11:25.285586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:11:25.285586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:11:25.285586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:11:25.287161 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:11:25.287161 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:11:25.287161 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:11:25.287161 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:11:25.287161 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:11:25.287161 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:11:25.287161 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:11:25.287161 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:11:25.287161 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:11:25.287161 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:11:25.287161 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:11:25.287161 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:11:25.287161 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 66/124] 10.222.0.87 () {56 vars in 1114 bytes} [Tue Jun 21 18:11:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:11:25.287934 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e0829-ce8e-4916-be07-ca368e3abcac None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:25.300324 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf44e6-76f9-4436-91fc-a35139526ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:25.302118 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf44e6-76f9-4436-91fc-a35139526ada None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:25.302635 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf44e6-76f9-4436-91fc-a35139526ada None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:25.303386 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf44e6-76f9-4436-91fc-a35139526ada None admin] PATH_INFO: `/v3/users/dd94cb8910014827bad3f45b205880f7` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:25.305806 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaf44e6-76f9-4436-91fc-a35139526ada None admin] RBAC: Authorizing `identity:get_user(user_id=dd94cb8910014827bad3f45b205880f7)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:25.308808 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaf44e6-76f9-4436-91fc-a35139526ada None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:25.312264 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 59/125] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:11:25 2022] GET /identity/v3/users/dd94cb8910014827bad3f45b205880f7 => generated 247 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:25.313961 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf44e6-76f9-4436-91fc-a35139526ada None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:25.383524 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:25.470205 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf44e6-76f9-4436-91fc-a35139526ada None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:25.496637 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e0829-ce8e-4916-be07-ca368e3abcac None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:25.543893 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d1bb9-4ddc-4b1b-9236-8744bc626ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:25.545195 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d1bb9-4ddc-4b1b-9236-8744bc626ea9 None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:25.545756 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d1bb9-4ddc-4b1b-9236-8744bc626ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:25.546255 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d1bb9-4ddc-4b1b-9236-8744bc626ea9 None admin] PATH_INFO: `/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f/users/786bcdb4dc274deb8280030ff16c6d0a/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:25.548858 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57d1bb9-4ddc-4b1b-9236-8744bc626ea9 None admin] RBAC: Authorizing `identity:create_grant(project_id=48caa0e9d7fe40d28e99abd1a7a6896f, user_id=786bcdb4dc274deb8280030ff16c6d0a, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:25.551523 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57d1bb9-4ddc-4b1b-9236-8744bc626ea9 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:25.559019 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5191ecb6-9081-4df4-ac7b-15fedb706c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:25.560745 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5191ecb6-9081-4df4-ac7b-15fedb706c4e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:25.561781 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5191ecb6-9081-4df4-ac7b-15fedb706c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:25.561921 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5191ecb6-9081-4df4-ac7b-15fedb706c4e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:25.572110 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 67/126] 10.222.0.87 () {58 vars in 1439 bytes} [Tue Jun 21 18:11:25 2022] PUT /identity/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f/users/786bcdb4dc274deb8280030ff16c6d0a/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:11:25.582401 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d1bb9-4ddc-4b1b-9236-8744bc626ea9 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:25.607474 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5191ecb6-9081-4df4-ac7b-15fedb706c4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:11:25.636694 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-5191ecb6-9081-4df4-ac7b-15fedb706c4e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:11:25.636694 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:11:25.636694 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:11:25.636694 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:11:25.636694 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:11:25.636694 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:11:25.636694 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:11:25.636694 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:11:25.636694 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:11:25.636694 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:11:25.636694 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:11:25.636694 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:11:25.636694 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:11:25.636694 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:11:25.636694 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:11:25.636694 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:11:25.637778 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:11:25.637778 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:11:25.637778 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:11:25.637778 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:11:25.637778 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:11:25.637778 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:11:25.637778 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:11:25.637778 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:11:25.637778 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:11:25.637778 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:11:25.637778 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:11:25.637778 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:11:25.637778 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:11:25.637778 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:11:25.637778 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:11:25.637778 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:11:25.638549 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:11:25.638549 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:11:25.638549 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:11:25.638549 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:11:25.638549 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:11:25.638549 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:11:25.638549 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:11:25.638549 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:11:25.638549 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:11:25.638549 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:11:25.638549 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:11:25.638549 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:11:25.639132 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 60/127] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:11:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:11:25.642523 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5191ecb6-9081-4df4-ac7b-15fedb706c4e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:25.806007 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:25.874380 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5191ecb6-9081-4df4-ac7b-15fedb706c4e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:25.919046 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d1bb9-4ddc-4b1b-9236-8744bc626ea9 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:25.947950 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b8a1fe-822c-424d-be98-f4417ef7f86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:25.948869 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8a1fe-822c-424d-be98-f4417ef7f86a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:25.949157 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8a1fe-822c-424d-be98-f4417ef7f86a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:25.949443 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8a1fe-822c-424d-be98-f4417ef7f86a None admin] PATH_INFO: `/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:25.950418 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:25.951056 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b8a1fe-822c-424d-be98-f4417ef7f86a None admin] RBAC: Authorizing `identity:get_project(project_id=48caa0e9d7fe40d28e99abd1a7a6896f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:25.954410 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b8a1fe-822c-424d-be98-f4417ef7f86a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:25.956705 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 61/128] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:11:25 2022] GET /identity/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f => generated 298 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:25.962689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b8a1fe-822c-424d-be98-f4417ef7f86a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:25.982060 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3761282-fe15-4f6e-91cc-e023b837dc91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:25.982769 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3761282-fe15-4f6e-91cc-e023b837dc91 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:25.983056 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3761282-fe15-4f6e-91cc-e023b837dc91 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:25.983337 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3761282-fe15-4f6e-91cc-e023b837dc91 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:25.984233 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3761282-fe15-4f6e-91cc-e023b837dc91 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:25.984552 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3761282-fe15-4f6e-91cc-e023b837dc91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:25.986522 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3761282-fe15-4f6e-91cc-e023b837dc91 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:25.997071 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 68/129] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:11:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 418 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:25.997993 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3761282-fe15-4f6e-91cc-e023b837dc91 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:26.142056 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b8a1fe-822c-424d-be98-f4417ef7f86a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:26.175900 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:26.186895 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d36f8-02f0-4d57-bdfa-aa36815bf3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:26.187961 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d36f8-02f0-4d57-bdfa-aa36815bf3a4 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:26.188801 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d36f8-02f0-4d57-bdfa-aa36815bf3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:26.189101 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d36f8-02f0-4d57-bdfa-aa36815bf3a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:26.189948 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5d36f8-02f0-4d57-bdfa-aa36815bf3a4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:26.190273 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5d36f8-02f0-4d57-bdfa-aa36815bf3a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:26.192068 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5d36f8-02f0-4d57-bdfa-aa36815bf3a4 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:26.193404 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3761282-fe15-4f6e-91cc-e023b837dc91 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:26.202187 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 62/130] 10.222.0.87 () {56 vars in 1101 bytes} [Tue Jun 21 18:11:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:26.204177 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d36f8-02f0-4d57-bdfa-aa36815bf3a4 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:26.293031 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cb217a-ae96-4b0f-a182-17226a26321c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:26.294043 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cb217a-ae96-4b0f-a182-17226a26321c None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:26.295582 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cb217a-ae96-4b0f-a182-17226a26321c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:26.295582 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cb217a-ae96-4b0f-a182-17226a26321c None admin] PATH_INFO: `/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f/users/dd94cb8910014827bad3f45b205880f7/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:26.297968 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cb217a-ae96-4b0f-a182-17226a26321c None admin] RBAC: Authorizing `identity:create_grant(project_id=48caa0e9d7fe40d28e99abd1a7a6896f, user_id=dd94cb8910014827bad3f45b205880f7, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:26.300450 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cb217a-ae96-4b0f-a182-17226a26321c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:26.327230 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 69/131] 10.222.0.87 () {58 vars in 1439 bytes} [Tue Jun 21 18:11:25 2022] PUT /identity/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f/users/dd94cb8910014827bad3f45b205880f7/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:11:26.327942 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cb217a-ae96-4b0f-a182-17226a26321c None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:26.384820 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d36f8-02f0-4d57-bdfa-aa36815bf3a4 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:26.403992 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:26.433510 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c83d2-e402-4e7c-b7d1-de2de26558ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:26.434932 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c83d2-e402-4e7c-b7d1-de2de26558ae None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:26.435441 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c83d2-e402-4e7c-b7d1-de2de26558ae None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:26.435869 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c83d2-e402-4e7c-b7d1-de2de26558ae None admin] PATH_INFO: `/v3/users/786bcdb4dc274deb8280030ff16c6d0a` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:26.437494 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8c83d2-e402-4e7c-b7d1-de2de26558ae None admin] RBAC: Authorizing `identity:get_user(user_id=786bcdb4dc274deb8280030ff16c6d0a)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:26.439415 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8c83d2-e402-4e7c-b7d1-de2de26558ae None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:26.441485 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 63/132] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:11:26 2022] GET /identity/v3/users/786bcdb4dc274deb8280030ff16c6d0a => generated 275 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:26.450385 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c83d2-e402-4e7c-b7d1-de2de26558ae None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:26.543800 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:26.600747 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cb217a-ae96-4b0f-a182-17226a26321c None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:26.603091 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c83d2-e402-4e7c-b7d1-de2de26558ae None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:26.645890 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d057c445-47a5-4fb8-ae64-4ee4c59a0acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:26.647387 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d057c445-47a5-4fb8-ae64-4ee4c59a0acf None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:26.647844 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d057c445-47a5-4fb8-ae64-4ee4c59a0acf None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:26.648284 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d057c445-47a5-4fb8-ae64-4ee4c59a0acf None admin] PATH_INFO: `/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:26.650152 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d057c445-47a5-4fb8-ae64-4ee4c59a0acf None admin] RBAC: Authorizing `identity:get_project(project_id=48caa0e9d7fe40d28e99abd1a7a6896f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:26.651911 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d057c445-47a5-4fb8-ae64-4ee4c59a0acf None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:26.654589 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 64/133] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:11:26 2022] GET /identity/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:26.663208 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d057c445-47a5-4fb8-ae64-4ee4c59a0acf None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:26.706920 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f14380-5112-4ef1-a9f2-fa575ca26842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:26.708288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f14380-5112-4ef1-a9f2-fa575ca26842 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:26.708747 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f14380-5112-4ef1-a9f2-fa575ca26842 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:26.709175 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f14380-5112-4ef1-a9f2-fa575ca26842 None admin] PATH_INFO: `/v3/users/786bcdb4dc274deb8280030ff16c6d0a` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:26.712313 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f14380-5112-4ef1-a9f2-fa575ca26842 None admin] RBAC: Authorizing `identity:get_user(user_id=786bcdb4dc274deb8280030ff16c6d0a)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:26.714447 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f14380-5112-4ef1-a9f2-fa575ca26842 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:26.716699 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 70/134] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:11:26 2022] GET /identity/v3/users/786bcdb4dc274deb8280030ff16c6d0a => generated 275 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:26.722677 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f14380-5112-4ef1-a9f2-fa575ca26842 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:26.906858 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d057c445-47a5-4fb8-ae64-4ee4c59a0acf None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:26.952074 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:26.955767 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f14380-5112-4ef1-a9f2-fa575ca26842 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:26.959287 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:26.968693 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39bf2f0-574d-4d79-bd93-81d92e749fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:26.969668 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39bf2f0-574d-4d79-bd93-81d92e749fb4 None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:26.970073 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39bf2f0-574d-4d79-bd93-81d92e749fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:26.970365 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39bf2f0-574d-4d79-bd93-81d92e749fb4 None admin] PATH_INFO: `/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f/users/786bcdb4dc274deb8280030ff16c6d0a/roles/46e77bb17fc3465496a7a3a29d210042` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:26.973802 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39bf2f0-574d-4d79-bd93-81d92e749fb4 None admin] RBAC: Authorizing `identity:create_grant(project_id=48caa0e9d7fe40d28e99abd1a7a6896f, user_id=786bcdb4dc274deb8280030ff16c6d0a, role_id=46e77bb17fc3465496a7a3a29d210042)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:26.975965 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39bf2f0-574d-4d79-bd93-81d92e749fb4 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:26.998460 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 65/135] 10.222.0.87 () {58 vars in 1439 bytes} [Tue Jun 21 18:11:26 2022] PUT /identity/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f/users/786bcdb4dc274deb8280030ff16c6d0a/roles/46e77bb17fc3465496a7a3a29d210042 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:11:27.000795 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23b2f2-a073-4a95-b485-8ed87e178129 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:27.001096 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23b2f2-a073-4a95-b485-8ed87e178129 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:27.001384 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23b2f2-a073-4a95-b485-8ed87e178129 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:27.003296 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 66/136] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:11:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:11:27.010027 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d0479-e031-4ed1-bb33-8cc995a695fc None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:27.010357 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d0479-e031-4ed1-bb33-8cc995a695fc None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:27.010663 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d0479-e031-4ed1-bb33-8cc995a695fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:27.012568 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3ed5a2-9a13-4f32-8cf3-4d4d4f60cf9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:27.013571 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3ed5a2-9a13-4f32-8cf3-4d4d4f60cf9d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:27.013906 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3ed5a2-9a13-4f32-8cf3-4d4d4f60cf9d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:27.014191 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3ed5a2-9a13-4f32-8cf3-4d4d4f60cf9d None admin] PATH_INFO: `/v3/projects/9abd12a809ab4ddfb059927f8acd84af` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:27.015634 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3ed5a2-9a13-4f32-8cf3-4d4d4f60cf9d None admin] RBAC: Authorizing `identity:get_project(project_id=9abd12a809ab4ddfb059927f8acd84af)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:27.017442 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3ed5a2-9a13-4f32-8cf3-4d4d4f60cf9d None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:27.020377 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 71/137] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:11:26 2022] GET /identity/v3/projects/9abd12a809ab4ddfb059927f8acd84af => generated 312 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:27.029012 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3ed5a2-9a13-4f32-8cf3-4d4d4f60cf9d None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:27.032073 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-713d0479-e031-4ed1-bb33-8cc995a695fc None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:11:27.033859 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-713d0479-e031-4ed1-bb33-8cc995a695fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:11:27.203313 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 67/138] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:11:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:11:27.252394 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3ed5a2-9a13-4f32-8cf3-4d4d4f60cf9d None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:27.305300 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b63b94b-68b1-4a89-83b3-d4e608517dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:27.306457 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b63b94b-68b1-4a89-83b3-d4e608517dac None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:27.306673 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b63b94b-68b1-4a89-83b3-d4e608517dac None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:27.306869 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b63b94b-68b1-4a89-83b3-d4e608517dac None admin] PATH_INFO: `/v3/projects/9abd12a809ab4ddfb059927f8acd84af/users/786bcdb4dc274deb8280030ff16c6d0a/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:27.309253 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b63b94b-68b1-4a89-83b3-d4e608517dac None admin] RBAC: Authorizing `identity:create_grant(project_id=9abd12a809ab4ddfb059927f8acd84af, user_id=786bcdb4dc274deb8280030ff16c6d0a, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:27.310837 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b63b94b-68b1-4a89-83b3-d4e608517dac None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:27.321938 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d0479-e031-4ed1-bb33-8cc995a695fc None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:27.354104 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 72/139] 10.222.0.87 () {58 vars in 1439 bytes} [Tue Jun 21 18:11:27 2022] PUT /identity/v3/projects/9abd12a809ab4ddfb059927f8acd84af/users/786bcdb4dc274deb8280030ff16c6d0a/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:11:27.505630 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d0479-e031-4ed1-bb33-8cc995a695fc None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:27.523096 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:27.560687 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2488e14-ae58-4e3e-a603-8f4882d97f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:27.561669 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2488e14-ae58-4e3e-a603-8f4882d97f2f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:27.563358 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2488e14-ae58-4e3e-a603-8f4882d97f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:27.563358 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2488e14-ae58-4e3e-a603-8f4882d97f2f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:27.564552 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2488e14-ae58-4e3e-a603-8f4882d97f2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:27.566085 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2488e14-ae58-4e3e-a603-8f4882d97f2f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:27.568060 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 68/140] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:11:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:27.576203 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b63b94b-68b1-4a89-83b3-d4e608517dac None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:27.629307 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:27.732802 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b63b94b-68b1-4a89-83b3-d4e608517dac None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:27.780963 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42311a99-a89f-4295-bfdc-ef88ca613e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:27.781679 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42311a99-a89f-4295-bfdc-ef88ca613e70 None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:27.781976 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42311a99-a89f-4295-bfdc-ef88ca613e70 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:27.782258 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42311a99-a89f-4295-bfdc-ef88ca613e70 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:27.783033 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42311a99-a89f-4295-bfdc-ef88ca613e70 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:27.784590 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42311a99-a89f-4295-bfdc-ef88ca613e70 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:27.825912 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 73/141] 10.222.0.87 () {60 vars in 1132 bytes} [Tue Jun 21 18:11:27 2022] POST /identity/v3/users => generated 283 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:11:28.002538 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7739aca3-cd5b-4401-a244-100f37f13033 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:28.002897 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7739aca3-cd5b-4401-a244-100f37f13033 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:28.003124 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7739aca3-cd5b-4401-a244-100f37f13033 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:28.005266 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 69/142] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:11:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:11:28.011469 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c7055-ef27-4611-951c-f4fe16e2e20b None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:28.011794 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c7055-ef27-4611-951c-f4fe16e2e20b None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:28.012049 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c7055-ef27-4611-951c-f4fe16e2e20b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:28.035634 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-363c7055-ef27-4611-951c-f4fe16e2e20b None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:11:28.036849 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-363c7055-ef27-4611-951c-f4fe16e2e20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:11:28.131146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 74/143] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:11:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:11:28.155088 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7739aca3-cd5b-4401-a244-100f37f13033 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:28.312820 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7739aca3-cd5b-4401-a244-100f37f13033 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:28.387677 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd97c1c-d890-4019-aced-7cb067717533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:28.389150 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd97c1c-d890-4019-aced-7cb067717533 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:28.389694 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd97c1c-d890-4019-aced-7cb067717533 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:28.390217 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd97c1c-d890-4019-aced-7cb067717533 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:28.405915 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd97c1c-d890-4019-aced-7cb067717533 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:11:28.414714 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-edd97c1c-d890-4019-aced-7cb067717533 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:11:28.414714 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:11:28.414714 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:11:28.414714 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:11:28.414714 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:11:28.414714 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:11:28.414714 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:11:28.414714 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:11:28.414714 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:11:28.414714 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:11:28.414714 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:11:28.414714 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:11:28.414714 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:11:28.414714 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:11:28.414714 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:11:28.414714 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:11:28.415706 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:11:28.415706 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:11:28.415706 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:11:28.415706 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:11:28.415706 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:11:28.415706 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:11:28.415706 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:11:28.415706 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:11:28.415706 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:11:28.415706 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:11:28.415706 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:11:28.415706 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:11:28.415706 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:11:28.415706 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:11:28.415706 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:11:28.415706 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:11:28.416510 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:11:28.416510 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:11:28.416510 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:11:28.416510 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:11:28.416510 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:11:28.416510 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:11:28.416510 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:11:28.416510 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:11:28.416510 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:11:28.416510 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:11:28.416510 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:11:28.416510 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:11:28.416510 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 70/144] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:11:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:11:28.422620 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c7055-ef27-4611-951c-f4fe16e2e20b None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:28.445024 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:28.566172 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c7055-ef27-4611-951c-f4fe16e2e20b None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:28.622757 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d2da21-6066-4801-ac7b-8c85b9f08f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:28.623495 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2da21-6066-4801-ac7b-8c85b9f08f19 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:28.623784 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2da21-6066-4801-ac7b-8c85b9f08f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:28.624066 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2da21-6066-4801-ac7b-8c85b9f08f19 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:28.624875 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d2da21-6066-4801-ac7b-8c85b9f08f19 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:28.625189 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d2da21-6066-4801-ac7b-8c85b9f08f19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:28.626713 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d2da21-6066-4801-ac7b-8c85b9f08f19 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:28.642761 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 75/145] 10.222.0.87 () {56 vars in 1101 bytes} [Tue Jun 21 18:11:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:28.650135 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd97c1c-d890-4019-aced-7cb067717533 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:28.714681 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:28.796789 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd97c1c-d890-4019-aced-7cb067717533 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:28.847661 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffa87fc-f76e-4d33-8083-ca3802aa293c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:28.849042 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa87fc-f76e-4d33-8083-ca3802aa293c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:28.849341 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa87fc-f76e-4d33-8083-ca3802aa293c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:28.849629 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa87fc-f76e-4d33-8083-ca3802aa293c None admin] PATH_INFO: `/v3/users/2ed789dda2154c888945486c0b8c55d3` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:28.851096 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffa87fc-f76e-4d33-8083-ca3802aa293c None admin] RBAC: Authorizing `identity:get_user(user_id=2ed789dda2154c888945486c0b8c55d3)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:28.853273 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffa87fc-f76e-4d33-8083-ca3802aa293c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:28.855305 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 71/146] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:11:28 2022] GET /identity/v3/users/2ed789dda2154c888945486c0b8c55d3 => generated 289 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:28.861752 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d2da21-6066-4801-ac7b-8c85b9f08f19 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:28.986765 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:29.023258 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d2da21-6066-4801-ac7b-8c85b9f08f19 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:29.075172 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db25a32-401d-4697-8ce1-882c02942245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:29.076106 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db25a32-401d-4697-8ce1-882c02942245 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:29.076386 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db25a32-401d-4697-8ce1-882c02942245 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:29.076658 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db25a32-401d-4697-8ce1-882c02942245 None admin] PATH_INFO: `/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:29.078043 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db25a32-401d-4697-8ce1-882c02942245 None admin] RBAC: Authorizing `identity:get_project(project_id=48caa0e9d7fe40d28e99abd1a7a6896f)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:29.079538 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db25a32-401d-4697-8ce1-882c02942245 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:29.081345 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 76/147] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:11:28 2022] GET /identity/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:11:29.087689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffa87fc-f76e-4d33-8083-ca3802aa293c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:29.142965 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c0a8d-6484-462f-8724-fecf951d2431 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:29.143140 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c0a8d-6484-462f-8724-fecf951d2431 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:29.143420 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c0a8d-6484-462f-8724-fecf951d2431 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:29.144866 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 77/148] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:11:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:11:29.162142 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81531705-0d88-4058-b51d-2803780ef517 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:29.162535 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81531705-0d88-4058-b51d-2803780ef517 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:29.162819 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81531705-0d88-4058-b51d-2803780ef517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:29.195774 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-81531705-0d88-4058-b51d-2803780ef517 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:11:29.197429 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-81531705-0d88-4058-b51d-2803780ef517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:11:29.306522 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffa87fc-f76e-4d33-8083-ca3802aa293c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:29.331816 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:29.371109 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 78/149] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:11:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:11:29.381726 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241d1872-8f70-46ce-af59-5c947014326e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:29.382682 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241d1872-8f70-46ce-af59-5c947014326e None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:29.383192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241d1872-8f70-46ce-af59-5c947014326e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:29.383692 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241d1872-8f70-46ce-af59-5c947014326e None admin] PATH_INFO: `/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f/users/2ed789dda2154c888945486c0b8c55d3/roles/5efb4931668c4edf87773a1f5d47f005` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:29.385949 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241d1872-8f70-46ce-af59-5c947014326e None admin] RBAC: Authorizing `identity:create_grant(project_id=48caa0e9d7fe40d28e99abd1a7a6896f, user_id=2ed789dda2154c888945486c0b8c55d3, role_id=5efb4931668c4edf87773a1f5d47f005)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:29.387624 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241d1872-8f70-46ce-af59-5c947014326e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:29.409062 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 72/150] 10.222.0.87 () {58 vars in 1439 bytes} [Tue Jun 21 18:11:29 2022] PUT /identity/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f/users/2ed789dda2154c888945486c0b8c55d3/roles/5efb4931668c4edf87773a1f5d47f005 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:11:29.455373 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81531705-0d88-4058-b51d-2803780ef517 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:29.566413 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:29.601691 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81531705-0d88-4058-b51d-2803780ef517 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:29.645052 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb548c-1e36-40d0-82cb-b29c4ffecf4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:29.646823 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb548c-1e36-40d0-82cb-b29c4ffecf4d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:29.647105 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb548c-1e36-40d0-82cb-b29c4ffecf4d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:29.647381 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb548c-1e36-40d0-82cb-b29c4ffecf4d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:29.669604 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeb548c-1e36-40d0-82cb-b29c4ffecf4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:11:29.679586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-8aeb548c-1e36-40d0-82cb-b29c4ffecf4d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:11:29.679586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:11:29.679586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:11:29.679586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:11:29.679586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:11:29.679586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:11:29.679586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:11:29.679586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:11:29.679586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:11:29.679586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:11:29.679586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:11:29.679586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:11:29.679586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:11:29.679586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:11:29.679586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:11:29.679586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:11:29.680563 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:11:29.680563 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:11:29.680563 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:11:29.680563 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:11:29.680563 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:11:29.680563 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:11:29.680563 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:11:29.680563 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:11:29.680563 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:11:29.680563 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:11:29.680563 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:11:29.680563 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:11:29.680563 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:11:29.680563 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:11:29.680563 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:11:29.680563 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:11:29.681331 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:11:29.681331 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:11:29.681331 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:11:29.681331 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:11:29.681331 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:11:29.681331 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:11:29.681331 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:11:29.681331 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:11:29.681331 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:11:29.681331 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:11:29.681331 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:11:29.681331 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:11:29.681331 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 79/151] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:11:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:11:29.694594 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241d1872-8f70-46ce-af59-5c947014326e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:29.848224 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241d1872-8f70-46ce-af59-5c947014326e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:29.893210 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c51d06-19a3-4e34-a045-46feb7294054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:29.893955 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c51d06-19a3-4e34-a045-46feb7294054 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:29.894272 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c51d06-19a3-4e34-a045-46feb7294054 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:29.894552 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c51d06-19a3-4e34-a045-46feb7294054 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:29.895363 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c51d06-19a3-4e34-a045-46feb7294054 None admin] RBAC: Adding query filter params (name=member) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:29.896462 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c51d06-19a3-4e34-a045-46feb7294054 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:29.898222 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c51d06-19a3-4e34-a045-46feb7294054 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:29.908959 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 73/152] 10.222.0.87 () {56 vars in 1101 bytes} [Tue Jun 21 18:11:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:29.915901 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb548c-1e36-40d0-82cb-b29c4ffecf4d None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:30.073692 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb548c-1e36-40d0-82cb-b29c4ffecf4d None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:30.120825 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8218bf1d-b426-4cef-87d3-5fe8a1a533dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:30.121817 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8218bf1d-b426-4cef-87d3-5fe8a1a533dc None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:30.122147 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8218bf1d-b426-4cef-87d3-5fe8a1a533dc None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:30.122425 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8218bf1d-b426-4cef-87d3-5fe8a1a533dc None admin] PATH_INFO: `/v3/users/786bcdb4dc274deb8280030ff16c6d0a` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:30.123775 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8218bf1d-b426-4cef-87d3-5fe8a1a533dc None admin] RBAC: Authorizing `identity:get_user(user_id=786bcdb4dc274deb8280030ff16c6d0a)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:30.125396 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8218bf1d-b426-4cef-87d3-5fe8a1a533dc None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:30.127632 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 80/153] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:11:29 2022] GET /identity/v3/users/786bcdb4dc274deb8280030ff16c6d0a => generated 275 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:30.134653 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c51d06-19a3-4e34-a045-46feb7294054 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:30.176540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:30.265049 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c51d06-19a3-4e34-a045-46feb7294054 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:30.306511 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013fb95e-ca83-4095-a26f-3947662e4a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:30.307246 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013fb95e-ca83-4095-a26f-3947662e4a4b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:30.307538 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013fb95e-ca83-4095-a26f-3947662e4a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:30.307819 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013fb95e-ca83-4095-a26f-3947662e4a4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:30.391559 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013fb95e-ca83-4095-a26f-3947662e4a4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:30.394933 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013fb95e-ca83-4095-a26f-3947662e4a4b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:30.438158 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:30.525807 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 74/154] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:11:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:11:30.527862 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8218bf1d-b426-4cef-87d3-5fe8a1a533dc None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:30.539968 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:30.671401 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8218bf1d-b426-4cef-87d3-5fe8a1a533dc None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:30.721875 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d1dc8b-2c94-44ed-bec8-70e7bea74082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:30.724182 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1dc8b-2c94-44ed-bec8-70e7bea74082 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:30.724676 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1dc8b-2c94-44ed-bec8-70e7bea74082 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:30.725158 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1dc8b-2c94-44ed-bec8-70e7bea74082 None admin] PATH_INFO: `/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:30.727603 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d1dc8b-2c94-44ed-bec8-70e7bea74082 None admin] RBAC: Authorizing `identity:get_project(project_id=48caa0e9d7fe40d28e99abd1a7a6896f)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:30.730446 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d1dc8b-2c94-44ed-bec8-70e7bea74082 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:30.734075 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 81/155] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:11:30 2022] GET /identity/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:30.741333 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013fb95e-ca83-4095-a26f-3947662e4a4b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:30.947952 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013fb95e-ca83-4095-a26f-3947662e4a4b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:31.010074 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:31.018526 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fea4db-1788-482b-afee-7acb65d62429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:31.020389 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fea4db-1788-482b-afee-7acb65d62429 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:31.022263 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fea4db-1788-482b-afee-7acb65d62429 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:31.023046 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fea4db-1788-482b-afee-7acb65d62429 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:31.025113 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fea4db-1788-482b-afee-7acb65d62429 None admin] RBAC: Adding query filter params (role.id=da6d4a0ab8c04809ba120170fb4ae8a9, scope.project.id=48caa0e9d7fe40d28e99abd1a7a6896f, user.id=786bcdb4dc274deb8280030ff16c6d0a) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:31.025720 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fea4db-1788-482b-afee-7acb65d62429 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:31.028721 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fea4db-1788-482b-afee-7acb65d62429 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:31.042078 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 75/156] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:11:30 2022] GET /identity/v3/role_assignments?user.id=786bcdb4dc274deb8280030ff16c6d0a&scope.project.id=48caa0e9d7fe40d28e99abd1a7a6896f&role.id=da6d4a0ab8c04809ba120170fb4ae8a9 => generated 608 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:31.591602 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf1c0e2-a5ff-4bca-b4af-f89b2743d82d None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:31.592171 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf1c0e2-a5ff-4bca-b4af-f89b2743d82d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:31.592542 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf1c0e2-a5ff-4bca-b4af-f89b2743d82d None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:31.595026 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 82/157] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:11:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:11:31.606949 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b01790d-feb5-4cc0-9a1f-7cce38787462 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:31.607460 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b01790d-feb5-4cc0-9a1f-7cce38787462 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:31.607987 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b01790d-feb5-4cc0-9a1f-7cce38787462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:31.639901 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-6b01790d-feb5-4cc0-9a1f-7cce38787462 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:11:31.642043 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6b01790d-feb5-4cc0-9a1f-7cce38787462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:11:31.755035 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 76/158] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:11:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:11:31.872560 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf1c0e2-a5ff-4bca-b4af-f89b2743d82d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:31.984226 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2118d73-e191-4bd4-bf7b-924abdacbdb1 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:31.984772 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2118d73-e191-4bd4-bf7b-924abdacbdb1 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:31.985563 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2118d73-e191-4bd4-bf7b-924abdacbdb1 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:31.988514 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 77/159] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:11:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:11:32.010979 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd1b5b5-7c7b-4f14-a742-ece6b642bf9d None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:32.011547 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd1b5b5-7c7b-4f14-a742-ece6b642bf9d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:32.012023 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd1b5b5-7c7b-4f14-a742-ece6b642bf9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:32.022113 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:32.075818 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-8cd1b5b5-7c7b-4f14-a742-ece6b642bf9d None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:11:32.077968 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8cd1b5b5-7c7b-4f14-a742-ece6b642bf9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:11:32.087315 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf1c0e2-a5ff-4bca-b4af-f89b2743d82d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:32.167659 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fa0e93-f8b3-4f28-b732-8d6f7e60a841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:32.168938 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa0e93-f8b3-4f28-b732-8d6f7e60a841 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:32.169410 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa0e93-f8b3-4f28-b732-8d6f7e60a841 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:32.169914 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa0e93-f8b3-4f28-b732-8d6f7e60a841 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:32.207313 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fa0e93-f8b3-4f28-b732-8d6f7e60a841 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:11:32.231158 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-b4fa0e93-f8b3-4f28-b732-8d6f7e60a841 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:11:32.231158 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:11:32.231158 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:11:32.231158 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:11:32.231158 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:11:32.231158 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:11:32.231158 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:11:32.231158 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:11:32.231158 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:11:32.231158 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:11:32.231158 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:11:32.231158 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:11:32.231158 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:11:32.231158 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:11:32.231158 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:11:32.231158 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:11:32.232124 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:11:32.232124 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:11:32.232124 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:11:32.232124 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:11:32.232124 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:11:32.232124 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:11:32.232124 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:11:32.232124 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:11:32.232124 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:11:32.232124 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:11:32.232124 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:11:32.232124 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:11:32.232124 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:11:32.232124 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:11:32.232124 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:11:32.232124 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:11:32.232960 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:11:32.232960 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:11:32.232960 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:11:32.232960 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:11:32.232960 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:11:32.232960 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:11:32.232960 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:11:32.232960 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:11:32.232960 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:11:32.232960 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:11:32.232960 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:11:32.232960 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:11:32.233541 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 83/160] 10.222.0.87 () {56 vars in 1096 bytes} [Tue Jun 21 18:11:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:11:32.251393 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fa0e93-f8b3-4f28-b732-8d6f7e60a841 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:32.272106 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 78/161] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:11:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:11:32.336840 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd1b5b5-7c7b-4f14-a742-ece6b642bf9d None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:32.498043 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:32.635178 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fa0e93-f8b3-4f28-b732-8d6f7e60a841 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:32.676768 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd1b5b5-7c7b-4f14-a742-ece6b642bf9d None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:32.753573 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:32.768640 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462aa885-4daf-42f7-a075-4057f6621522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:32.769214 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60bf38d-8367-43fa-9961-896947d4f0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:32.769509 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462aa885-4daf-42f7-a075-4057f6621522 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:32.769837 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462aa885-4daf-42f7-a075-4057f6621522 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:32.769976 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60bf38d-8367-43fa-9961-896947d4f0e6 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:32.770206 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462aa885-4daf-42f7-a075-4057f6621522 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:32.770436 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60bf38d-8367-43fa-9961-896947d4f0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:32.770916 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60bf38d-8367-43fa-9961-896947d4f0e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:32.772347 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60bf38d-8367-43fa-9961-896947d4f0e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:32.772904 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60bf38d-8367-43fa-9961-896947d4f0e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:32.775674 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60bf38d-8367-43fa-9961-896947d4f0e6 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:32.800578 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462aa885-4daf-42f7-a075-4057f6621522 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:11:32.804827 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 84/162] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:11:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 555 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:32.807210 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a8458-d4bf-4cbc-b3f2-f08cd4a40dc1 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:32.807717 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a8458-d4bf-4cbc-b3f2-f08cd4a40dc1 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:32.808209 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a8458-d4bf-4cbc-b3f2-f08cd4a40dc1 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:32.819114 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 85/163] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:11:32 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:11:32.820590 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a8458-d4bf-4cbc-b3f2-f08cd4a40dc1 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:32.822192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-462aa885-4daf-42f7-a075-4057f6621522 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:11:32.822192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:11:32.822192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:11:32.822192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:11:32.822192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:11:32.822192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:11:32.822192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:11:32.822192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:11:32.822192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:11:32.822192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:11:32.822192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:11:32.822192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:11:32.822192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:11:32.822192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:11:32.822192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:11:32.822192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:11:32.823051 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:11:32.823051 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:11:32.823051 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:11:32.823051 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:11:32.823051 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:11:32.823051 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:11:32.823051 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:11:32.823051 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:11:32.823051 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:11:32.823051 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:11:32.823051 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:11:32.823051 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:11:32.823051 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:11:32.823051 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:11:32.823051 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:11:32.823051 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:11:32.824334 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:11:32.824334 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:11:32.824334 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:11:32.824334 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:11:32.824334 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:11:32.824334 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:11:32.824334 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:11:32.824334 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:11:32.824334 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:11:32.824334 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:11:32.824334 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:11:32.824334 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:11:32.824334 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 79/164] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:11:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 487 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:11:32.832418 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462aa885-4daf-42f7-a075-4057f6621522 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:33.074452 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462aa885-4daf-42f7-a075-4057f6621522 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:33.113295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:33.160775 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83678883-7a0f-4561-8322-3ecd720a4491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:33.161927 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83678883-7a0f-4561-8322-3ecd720a4491 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:33.162296 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83678883-7a0f-4561-8322-3ecd720a4491 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:33.162665 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83678883-7a0f-4561-8322-3ecd720a4491 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:33.163519 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83678883-7a0f-4561-8322-3ecd720a4491 None admin] RBAC: Adding query filter params (name=member) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:33.163858 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83678883-7a0f-4561-8322-3ecd720a4491 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:33.165311 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83678883-7a0f-4561-8322-3ecd720a4491 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:33.191317 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a8458-d4bf-4cbc-b3f2-f08cd4a40dc1 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:33.194090 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 80/165] 10.222.0.87 () {56 vars in 1101 bytes} [Tue Jun 21 18:11:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:33.197118 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54063cf-8676-4864-b155-fd4e3b737705 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:33.197404 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54063cf-8676-4864-b155-fd4e3b737705 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:33.197710 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54063cf-8676-4864-b155-fd4e3b737705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:33.243308 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d54063cf-8676-4864-b155-fd4e3b737705 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:11:33.245141 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d54063cf-8676-4864-b155-fd4e3b737705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:11:33.259919 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b024a9f1-1c84-437d-b8a5-0b4d06b8cbf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:33.261388 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b024a9f1-1c84-437d-b8a5-0b4d06b8cbf8 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:33.261700 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b024a9f1-1c84-437d-b8a5-0b4d06b8cbf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:33.262018 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b024a9f1-1c84-437d-b8a5-0b4d06b8cbf8 None admin] PATH_INFO: `/v3/users/dd94cb8910014827bad3f45b205880f7` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:33.263760 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b024a9f1-1c84-437d-b8a5-0b4d06b8cbf8 None admin] RBAC: Authorizing `identity:get_user(user_id=dd94cb8910014827bad3f45b205880f7)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:33.265350 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b024a9f1-1c84-437d-b8a5-0b4d06b8cbf8 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:33.267231 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 86/166] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:11:32 2022] GET /identity/v3/users/dd94cb8910014827bad3f45b205880f7 => generated 247 bytes in 448 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:33.268070 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b024a9f1-1c84-437d-b8a5-0b4d06b8cbf8 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:33.356171 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:33.407231 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b024a9f1-1c84-437d-b8a5-0b4d06b8cbf8 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:33.453687 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc67b4c-ac51-445a-aef2-b00d04d666c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:33.453687 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc67b4c-ac51-445a-aef2-b00d04d666c0 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:33.453687 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc67b4c-ac51-445a-aef2-b00d04d666c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:33.453687 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc67b4c-ac51-445a-aef2-b00d04d666c0 None admin] PATH_INFO: `/v3/users/786bcdb4dc274deb8280030ff16c6d0a` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:33.457725 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc67b4c-ac51-445a-aef2-b00d04d666c0 None admin] RBAC: Authorizing `identity:get_user(user_id=786bcdb4dc274deb8280030ff16c6d0a)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:33.459816 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc67b4c-ac51-445a-aef2-b00d04d666c0 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:33.463698 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 87/167] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:11:33 2022] GET /identity/v3/users/786bcdb4dc274deb8280030ff16c6d0a => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:33.469906 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc67b4c-ac51-445a-aef2-b00d04d666c0 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:33.475463 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 81/168] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:11:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:11:33.481666 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867bf86-a694-4bd6-b755-dc2c8b707b54 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:33.481666 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867bf86-a694-4bd6-b755-dc2c8b707b54 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:33.481666 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867bf86-a694-4bd6-b755-dc2c8b707b54 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:33.481666 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 82/169] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:11:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:11:33.495369 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db30e7da-21e0-400c-a26f-0294c8e1dd13 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:33.495658 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db30e7da-21e0-400c-a26f-0294c8e1dd13 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:33.495926 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db30e7da-21e0-400c-a26f-0294c8e1dd13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:33.538125 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-db30e7da-21e0-400c-a26f-0294c8e1dd13 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:11:33.539957 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-db30e7da-21e0-400c-a26f-0294c8e1dd13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:11:33.715176 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 83/170] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:11:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:11:33.718626 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db30e7da-21e0-400c-a26f-0294c8e1dd13 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:33.768484 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc67b4c-ac51-445a-aef2-b00d04d666c0 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:33.798162 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:33.816924 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:33.825361 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d77ed75-b2ca-43f2-b5e9-bfefc8d52d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:33.826389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77ed75-b2ca-43f2-b5e9-bfefc8d52d9e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:33.826832 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77ed75-b2ca-43f2-b5e9-bfefc8d52d9e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:33.827259 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77ed75-b2ca-43f2-b5e9-bfefc8d52d9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:33.907479 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db30e7da-21e0-400c-a26f-0294c8e1dd13 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:33.982410 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d77ed75-b2ca-43f2-b5e9-bfefc8d52d9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:33.984187 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d77ed75-b2ca-43f2-b5e9-bfefc8d52d9e None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:33.988713 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e0b779-7c65-4375-9c16-6146f6ea2323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:33.990382 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0b779-7c65-4375-9c16-6146f6ea2323 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:33.990866 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0b779-7c65-4375-9c16-6146f6ea2323 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:33.991169 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0b779-7c65-4375-9c16-6146f6ea2323 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:34.140896 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e0b779-7c65-4375-9c16-6146f6ea2323 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:34.143942 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e0b779-7c65-4375-9c16-6146f6ea2323 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:34.177215 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 88/171] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:11:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 713 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:11:34.179268 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d77ed75-b2ca-43f2-b5e9-bfefc8d52d9e None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:34.220479 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:34.257742 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:34.344345 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 84/172] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:11:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 628 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:11:34.346226 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f98c3-f808-45b2-bc3c-c7ea1b9e857c None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:34.346712 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f98c3-f808-45b2-bc3c-c7ea1b9e857c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:34.347154 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f98c3-f808-45b2-bc3c-c7ea1b9e857c None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:34.348835 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 85/173] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:11:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:11:34.349813 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f98c3-f808-45b2-bc3c-c7ea1b9e857c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:34.384417 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d77ed75-b2ca-43f2-b5e9-bfefc8d52d9e None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:34.453080 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668c4805-37e7-489f-a407-309ae0588963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:34.453960 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668c4805-37e7-489f-a407-309ae0588963 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:34.454245 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668c4805-37e7-489f-a407-309ae0588963 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:34.454514 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668c4805-37e7-489f-a407-309ae0588963 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:34.472428 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668c4805-37e7-489f-a407-309ae0588963 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:11:34.481920 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-668c4805-37e7-489f-a407-309ae0588963 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:11:34.481920 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:11:34.481920 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:11:34.481920 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:11:34.481920 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:11:34.481920 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:11:34.481920 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:11:34.481920 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:11:34.481920 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:11:34.481920 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:11:34.481920 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:11:34.481920 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:11:34.481920 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:11:34.481920 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:11:34.481920 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:11:34.481920 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:11:34.483042 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:11:34.483042 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:11:34.483042 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:11:34.483042 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:11:34.483042 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:11:34.483042 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:11:34.483042 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:11:34.483042 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:11:34.483042 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:11:34.483042 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:11:34.483042 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:11:34.483042 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:11:34.483042 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:11:34.483042 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:11:34.483042 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:11:34.483042 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:11:34.486118 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:11:34.486118 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:11:34.486118 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:11:34.486118 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:11:34.486118 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:11:34.486118 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:11:34.486118 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:11:34.486118 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:11:34.486118 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:11:34.486118 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:11:34.486118 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:11:34.486118 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:11:34.486118 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 89/174] 10.222.0.87 () {56 vars in 1114 bytes} [Tue Jun 21 18:11:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:11:34.486118 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668c4805-37e7-489f-a407-309ae0588963 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:34.503765 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f98c3-f808-45b2-bc3c-c7ea1b9e857c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:34.557263 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b030e3-c5b9-489c-9d03-06c2d14791fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:34.558107 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b030e3-c5b9-489c-9d03-06c2d14791fe None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:34.558481 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b030e3-c5b9-489c-9d03-06c2d14791fe None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:34.558769 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b030e3-c5b9-489c-9d03-06c2d14791fe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:34.567104 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:34.571812 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b030e3-c5b9-489c-9d03-06c2d14791fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:11:34.580406 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-d6b030e3-c5b9-489c-9d03-06c2d14791fe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:11:34.580406 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:11:34.580406 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:11:34.580406 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:11:34.580406 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:11:34.580406 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:11:34.580406 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:11:34.580406 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:11:34.580406 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:11:34.580406 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:11:34.580406 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:11:34.580406 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:11:34.580406 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:11:34.580406 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:11:34.580406 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:11:34.580406 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:11:34.581544 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:11:34.581544 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:11:34.581544 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:11:34.581544 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:11:34.581544 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:11:34.581544 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:11:34.581544 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:11:34.581544 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:11:34.581544 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:11:34.581544 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:11:34.581544 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:11:34.581544 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:11:34.581544 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:11:34.581544 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:11:34.581544 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:11:34.581544 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:11:34.582540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:11:34.582540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:11:34.582540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:11:34.582540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:11:34.582540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:11:34.582540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:11:34.582540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:11:34.582540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:11:34.582540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:11:34.582540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:11:34.582540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:11:34.582540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:11:34.582540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 86/175] 10.222.0.87 () {56 vars in 1096 bytes} [Tue Jun 21 18:11:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:11:34.583510 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997cd39-03f8-45cc-9407-874d7dc1e576 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:34.583510 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997cd39-03f8-45cc-9407-874d7dc1e576 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:34.583766 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997cd39-03f8-45cc-9407-874d7dc1e576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:34.604030 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-6997cd39-03f8-45cc-9407-874d7dc1e576 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:11:34.605441 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6997cd39-03f8-45cc-9407-874d7dc1e576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:11:34.644578 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:34.671656 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668c4805-37e7-489f-a407-309ae0588963 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:34.697896 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 87/176] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:11:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:11:34.700889 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6997cd39-03f8-45cc-9407-874d7dc1e576 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:34.743913 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e789c851-21f4-4d55-b0e9-bcc6bf13e4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:34.744877 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e789c851-21f4-4d55-b0e9-bcc6bf13e4bb None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:34.745165 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e789c851-21f4-4d55-b0e9-bcc6bf13e4bb None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:34.745444 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e789c851-21f4-4d55-b0e9-bcc6bf13e4bb None admin] PATH_INFO: `/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:34.747081 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e789c851-21f4-4d55-b0e9-bcc6bf13e4bb None admin] RBAC: Authorizing `identity:get_project(project_id=48caa0e9d7fe40d28e99abd1a7a6896f)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:34.748678 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e789c851-21f4-4d55-b0e9-bcc6bf13e4bb None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:34.750699 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 90/177] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:11:34 2022] GET /identity/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f => generated 298 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:34.757351 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0282cb-7be7-4c2d-bf3b-89f5d5dedcb2 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:34.757351 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0282cb-7be7-4c2d-bf3b-89f5d5dedcb2 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:34.757351 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0282cb-7be7-4c2d-bf3b-89f5d5dedcb2 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:34.757351 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 91/178] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:11:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:11:34.757351 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0282cb-7be7-4c2d-bf3b-89f5d5dedcb2 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:34.894516 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6997cd39-03f8-45cc-9407-874d7dc1e576 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:34.916946 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0282cb-7be7-4c2d-bf3b-89f5d5dedcb2 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:34.934025 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:34.944155 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b5d997-b3ba-4e11-9cbf-c05dfedafa21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:34.945211 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5d997-b3ba-4e11-9cbf-c05dfedafa21 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:34.945561 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5d997-b3ba-4e11-9cbf-c05dfedafa21 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:34.945969 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5d997-b3ba-4e11-9cbf-c05dfedafa21 None admin] PATH_INFO: `/v3/projects/9abd12a809ab4ddfb059927f8acd84af` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:34.947548 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b5d997-b3ba-4e11-9cbf-c05dfedafa21 None admin] RBAC: Authorizing `identity:get_project(project_id=9abd12a809ab4ddfb059927f8acd84af)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:34.949103 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b5d997-b3ba-4e11-9cbf-c05dfedafa21 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:34.951206 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 88/179] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:11:34 2022] GET /identity/v3/projects/9abd12a809ab4ddfb059927f8acd84af => generated 312 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:34.952533 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e57b79-0c06-4843-a5bf-31e594b03399 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:34.952903 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e57b79-0c06-4843-a5bf-31e594b03399 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:34.953188 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e57b79-0c06-4843-a5bf-31e594b03399 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:34.954591 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 89/180] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:11:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:11:34.955325 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e57b79-0c06-4843-a5bf-31e594b03399 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:34.978951 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69cd9f6-4f34-4728-8f90-16bcb844269a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:34.979655 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69cd9f6-4f34-4728-8f90-16bcb844269a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:34.979940 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69cd9f6-4f34-4728-8f90-16bcb844269a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:34.980218 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69cd9f6-4f34-4728-8f90-16bcb844269a None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:34.980993 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69cd9f6-4f34-4728-8f90-16bcb844269a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:34.981301 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69cd9f6-4f34-4728-8f90-16bcb844269a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:34.982797 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69cd9f6-4f34-4728-8f90-16bcb844269a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:34.993713 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 92/181] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:11:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:34.994624 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69cd9f6-4f34-4728-8f90-16bcb844269a None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:35.028242 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:35.113354 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e57b79-0c06-4843-a5bf-31e594b03399 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:35.126858 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69cd9f6-4f34-4728-8f90-16bcb844269a None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:35.170667 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0183f0-1d42-4ca6-a1c0-de8c9b8efbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:35.171748 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0183f0-1d42-4ca6-a1c0-de8c9b8efbfb None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:35.172206 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0183f0-1d42-4ca6-a1c0-de8c9b8efbfb None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:35.172665 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0183f0-1d42-4ca6-a1c0-de8c9b8efbfb None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:35.174071 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0183f0-1d42-4ca6-a1c0-de8c9b8efbfb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:35.174589 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0183f0-1d42-4ca6-a1c0-de8c9b8efbfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:35.175358 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596287c9-0adf-436b-b59e-d7f8bfc5183c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:35.176648 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596287c9-0adf-436b-b59e-d7f8bfc5183c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:35.176989 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0183f0-1d42-4ca6-a1c0-de8c9b8efbfb None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:35.177089 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596287c9-0adf-436b-b59e-d7f8bfc5183c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:35.177534 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596287c9-0adf-436b-b59e-d7f8bfc5183c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:35.193392 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 90/182] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:11:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:35.194790 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0183f0-1d42-4ca6-a1c0-de8c9b8efbfb None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:35.195987 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596287c9-0adf-436b-b59e-d7f8bfc5183c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:11:35.209847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-596287c9-0adf-436b-b59e-d7f8bfc5183c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:11:35.209847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:11:35.209847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:11:35.209847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:11:35.209847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:11:35.209847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:11:35.209847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:11:35.209847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:11:35.209847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:11:35.209847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:11:35.209847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:11:35.209847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:11:35.209847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:11:35.209847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:11:35.209847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:11:35.209847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:11:35.211380 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:11:35.211380 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:11:35.211380 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:11:35.211380 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:11:35.211380 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:11:35.211380 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:11:35.211380 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:11:35.211380 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:11:35.211380 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:11:35.211380 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:11:35.211380 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:11:35.211380 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:11:35.211380 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:11:35.211380 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:11:35.211380 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:11:35.211380 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:11:35.212241 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:11:35.212241 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:11:35.212241 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:11:35.212241 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:11:35.212241 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:11:35.212241 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:11:35.212241 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:11:35.212241 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:11:35.212241 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:11:35.212241 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:11:35.212241 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:11:35.212241 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:11:35.212241 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 93/183] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:11:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:11:35.213960 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676c985c-5d54-4c0f-8bdc-3e2930cbf568 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:35.214449 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676c985c-5d54-4c0f-8bdc-3e2930cbf568 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:35.214930 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676c985c-5d54-4c0f-8bdc-3e2930cbf568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:35.237581 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-676c985c-5d54-4c0f-8bdc-3e2930cbf568 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:11:35.239061 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-676c985c-5d54-4c0f-8bdc-3e2930cbf568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:11:35.290376 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:35.338389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 94/184] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:11:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:11:35.349117 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b95b8-0d02-4139-8d1d-66165a8ffd65 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:35.349117 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b95b8-0d02-4139-8d1d-66165a8ffd65 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:35.349117 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b95b8-0d02-4139-8d1d-66165a8ffd65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:35.366045 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0183f0-1d42-4ca6-a1c0-de8c9b8efbfb None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:35.369913 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-d33b95b8-0d02-4139-8d1d-66165a8ffd65 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:11:35.371460 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d33b95b8-0d02-4139-8d1d-66165a8ffd65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:11:35.378309 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:35.428914 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a94ef99-de49-455b-8e60-0b42deb23ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:35.429889 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a94ef99-de49-455b-8e60-0b42deb23ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:35.430238 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a94ef99-de49-455b-8e60-0b42deb23ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:35.430575 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a94ef99-de49-455b-8e60-0b42deb23ed4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:35.431766 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a94ef99-de49-455b-8e60-0b42deb23ed4 None admin] RBAC: Adding query filter params (role.id=98f0943a610f4a9e9f8de54f6e9f717d, scope.project.id=48caa0e9d7fe40d28e99abd1a7a6896f, user.id=dd94cb8910014827bad3f45b205880f7) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:35.432148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a94ef99-de49-455b-8e60-0b42deb23ed4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:35.433770 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a94ef99-de49-455b-8e60-0b42deb23ed4 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:35.443069 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 91/185] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:11:35 2022] GET /identity/v3/role_assignments?user.id=dd94cb8910014827bad3f45b205880f7&scope.project.id=48caa0e9d7fe40d28e99abd1a7a6896f&role.id=98f0943a610f4a9e9f8de54f6e9f717d => generated 608 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:35.444642 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a94ef99-de49-455b-8e60-0b42deb23ed4 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:35.521689 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 95/186] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:11:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:11:35.521689 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b95b8-0d02-4139-8d1d-66165a8ffd65 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:35.666112 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a94ef99-de49-455b-8e60-0b42deb23ed4 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:35.698006 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b95b8-0d02-4139-8d1d-66165a8ffd65 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:35.708994 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:35.727980 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a7298-84f3-40b4-bba5-7bae87fe5c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:35.729142 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a7298-84f3-40b4-bba5-7bae87fe5c01 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:35.729753 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a7298-84f3-40b4-bba5-7bae87fe5c01 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:35.730186 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a7298-84f3-40b4-bba5-7bae87fe5c01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:35.731883 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0a7298-84f3-40b4-bba5-7bae87fe5c01 None admin] RBAC: Adding query filter params (role.id=da6d4a0ab8c04809ba120170fb4ae8a9, scope.project.id=9abd12a809ab4ddfb059927f8acd84af, user.id=786bcdb4dc274deb8280030ff16c6d0a) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:35.732460 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0a7298-84f3-40b4-bba5-7bae87fe5c01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:35.736023 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0a7298-84f3-40b4-bba5-7bae87fe5c01 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:35.748105 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 92/187] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:11:35 2022] GET /identity/v3/role_assignments?user.id=786bcdb4dc274deb8280030ff16c6d0a&scope.project.id=9abd12a809ab4ddfb059927f8acd84af&role.id=da6d4a0ab8c04809ba120170fb4ae8a9 => generated 608 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:35.749618 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a7298-84f3-40b4-bba5-7bae87fe5c01 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:35.758242 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a770510c-6df4-4616-9529-e5a0be7454ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:35.759872 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a770510c-6df4-4616-9529-e5a0be7454ab None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:35.760354 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a770510c-6df4-4616-9529-e5a0be7454ab None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:35.760836 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a770510c-6df4-4616-9529-e5a0be7454ab None admin] PATH_INFO: `/v3/users/786bcdb4dc274deb8280030ff16c6d0a` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:35.763323 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a770510c-6df4-4616-9529-e5a0be7454ab None admin] RBAC: Authorizing `identity:get_user(user_id=786bcdb4dc274deb8280030ff16c6d0a)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:35.766315 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a770510c-6df4-4616-9529-e5a0be7454ab None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:35.769987 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 96/188] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:11:35 2022] GET /identity/v3/users/786bcdb4dc274deb8280030ff16c6d0a => generated 275 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:35.771384 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a770510c-6df4-4616-9529-e5a0be7454ab None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:35.793201 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:35.941118 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a7298-84f3-40b4-bba5-7bae87fe5c01 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:35.956772 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a770510c-6df4-4616-9529-e5a0be7454ab None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:36.011121 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce6576-1333-4854-992b-38336dfa3f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:36.012927 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce6576-1333-4854-992b-38336dfa3f40 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:36.013209 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce6576-1333-4854-992b-38336dfa3f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:36.013769 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce6576-1333-4854-992b-38336dfa3f40 None admin] PATH_INFO: `/v3/users/cd79af98206947fdb9a546a7cb70e1a6` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:36.015965 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1e08a4-e528-4f64-ab60-5c2822bc3761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:36.016667 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1e08a4-e528-4f64-ab60-5c2822bc3761 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:36.016941 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1e08a4-e528-4f64-ab60-5c2822bc3761 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:36.017516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1e08a4-e528-4f64-ab60-5c2822bc3761 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:36.018313 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1e08a4-e528-4f64-ab60-5c2822bc3761 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:36.018622 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1e08a4-e528-4f64-ab60-5c2822bc3761 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:36.023951 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1e08a4-e528-4f64-ab60-5c2822bc3761 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:36.032920 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 97/189] 10.222.0.87 () {56 vars in 1101 bytes} [Tue Jun 21 18:11:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:36.035659 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1e08a4-e528-4f64-ab60-5c2822bc3761 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:36.041214 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce6576-1333-4854-992b-38336dfa3f40 None admin] RBAC: Authorizing `identity:get_user(user_id=cd79af98206947fdb9a546a7cb70e1a6)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:36.044944 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce6576-1333-4854-992b-38336dfa3f40 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:36.049961 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 93/190] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:11:35 2022] GET /identity/v3/users/cd79af98206947fdb9a546a7cb70e1a6 => generated 283 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:36.049961 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce6576-1333-4854-992b-38336dfa3f40 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:36.129288 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:36.144778 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:36.182297 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1e08a4-e528-4f64-ab60-5c2822bc3761 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:36.201781 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce6576-1333-4854-992b-38336dfa3f40 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:36.226799 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f54067-e855-44df-a8ac-75fd023b9728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:36.228077 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f54067-e855-44df-a8ac-75fd023b9728 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:36.228522 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f54067-e855-44df-a8ac-75fd023b9728 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:36.228975 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f54067-e855-44df-a8ac-75fd023b9728 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:36.230962 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f54067-e855-44df-a8ac-75fd023b9728 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:36.233610 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f54067-e855-44df-a8ac-75fd023b9728 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:36.236918 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 98/191] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:11:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:36.238583 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f54067-e855-44df-a8ac-75fd023b9728 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:36.255750 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8064654d-76b5-437e-9ea4-774d7f8bae34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:36.256776 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064654d-76b5-437e-9ea4-774d7f8bae34 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:36.257230 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064654d-76b5-437e-9ea4-774d7f8bae34 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:36.257599 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064654d-76b5-437e-9ea4-774d7f8bae34 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:36.272648 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8064654d-76b5-437e-9ea4-774d7f8bae34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:11:36.282114 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-8064654d-76b5-437e-9ea4-774d7f8bae34 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:11:36.282114 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:11:36.282114 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:11:36.282114 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:11:36.282114 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:11:36.282114 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:11:36.282114 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:11:36.282114 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:11:36.282114 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:11:36.282114 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:11:36.282114 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:11:36.282114 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:11:36.282114 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:11:36.282114 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:11:36.282114 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:11:36.282114 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:11:36.283901 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:11:36.283901 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:11:36.283901 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:11:36.283901 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:11:36.283901 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:11:36.283901 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:11:36.283901 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:11:36.283901 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:11:36.283901 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:11:36.283901 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:11:36.283901 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:11:36.283901 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:11:36.283901 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:11:36.283901 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:11:36.283901 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:11:36.283901 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:11:36.286471 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:11:36.286471 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:11:36.286471 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:11:36.286471 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:11:36.286471 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:11:36.286471 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:11:36.286471 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:11:36.286471 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:11:36.286471 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:11:36.286471 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:11:36.286471 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:11:36.286471 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:11:36.286471 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 94/192] 10.222.0.87 () {56 vars in 1114 bytes} [Tue Jun 21 18:11:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:11:36.286471 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8064654d-76b5-437e-9ea4-774d7f8bae34 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:36.383778 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f54067-e855-44df-a8ac-75fd023b9728 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:36.435747 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ad20a7-a4ba-4c99-a76e-24a0b4699ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:36.436575 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad20a7-a4ba-4c99-a76e-24a0b4699ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:36.436947 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad20a7-a4ba-4c99-a76e-24a0b4699ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:36.437367 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad20a7-a4ba-4c99-a76e-24a0b4699ba2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:36.439564 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8064654d-76b5-437e-9ea4-774d7f8bae34 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:36.442716 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:36.452805 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:36.497584 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f1bd6e-d0e6-4f6e-9665-71f5e0f14402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:36.498752 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1bd6e-d0e6-4f6e-9665-71f5e0f14402 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:36.499125 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1bd6e-d0e6-4f6e-9665-71f5e0f14402 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:36.499492 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1bd6e-d0e6-4f6e-9665-71f5e0f14402 None admin] PATH_INFO: `/v3/users/2ed789dda2154c888945486c0b8c55d3` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:36.501028 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f1bd6e-d0e6-4f6e-9665-71f5e0f14402 None admin] RBAC: Authorizing `identity:get_user(user_id=2ed789dda2154c888945486c0b8c55d3)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:36.502987 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f1bd6e-d0e6-4f6e-9665-71f5e0f14402 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:36.505223 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 95/193] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:11:36 2022] GET /identity/v3/users/2ed789dda2154c888945486c0b8c55d3 => generated 289 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:36.506531 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f1bd6e-d0e6-4f6e-9665-71f5e0f14402 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:36.551810 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ad20a7-a4ba-4c99-a76e-24a0b4699ba2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:36.553516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ad20a7-a4ba-4c99-a76e-24a0b4699ba2 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:36.658504 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f1bd6e-d0e6-4f6e-9665-71f5e0f14402 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:36.691190 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 99/194] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:11:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:11:36.692721 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ad20a7-a4ba-4c99-a76e-24a0b4699ba2 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:36.710839 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c15f4b-82df-478c-aa87-5361b421492b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:36.712149 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c15f4b-82df-478c-aa87-5361b421492b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:36.712957 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c15f4b-82df-478c-aa87-5361b421492b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:36.713104 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c15f4b-82df-478c-aa87-5361b421492b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:36.755508 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:36.773306 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:36.816930 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c15f4b-82df-478c-aa87-5361b421492b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:36.819772 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c15f4b-82df-478c-aa87-5361b421492b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:36.875140 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ad20a7-a4ba-4c99-a76e-24a0b4699ba2 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:36.932871 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d8ae64-09e1-4b85-b4eb-f0f251b0a239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:36.933732 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d8ae64-09e1-4b85-b4eb-f0f251b0a239 None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:36.933879 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d8ae64-09e1-4b85-b4eb-f0f251b0a239 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:36.934137 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d8ae64-09e1-4b85-b4eb-f0f251b0a239 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:36.935052 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d8ae64-09e1-4b85-b4eb-f0f251b0a239 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:36.936789 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d8ae64-09e1-4b85-b4eb-f0f251b0a239 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:36.962832 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 100/195] 10.222.0.87 () {60 vars in 1132 bytes} [Tue Jun 21 18:11:36 2022] POST /identity/v3/users => generated 297 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:11:36.963795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d8ae64-09e1-4b85-b4eb-f0f251b0a239 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:37.006597 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 96/196] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:11:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:11:37.006597 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c15f4b-82df-478c-aa87-5361b421492b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:37.120755 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:37.167608 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c15f4b-82df-478c-aa87-5361b421492b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:37.175506 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:37.188291 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d8ae64-09e1-4b85-b4eb-f0f251b0a239 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:37.222965 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709ada31-8ddc-4c1c-8567-fb443957cd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:37.224404 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ada31-8ddc-4c1c-8567-fb443957cd56 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:37.224404 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ada31-8ddc-4c1c-8567-fb443957cd56 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:37.224404 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ada31-8ddc-4c1c-8567-fb443957cd56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:37.260222 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b1e00-ea27-49aa-b4e7-47951cb06d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:37.261726 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b1e00-ea27-49aa-b4e7-47951cb06d81 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:37.262775 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b1e00-ea27-49aa-b4e7-47951cb06d81 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:37.263554 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b1e00-ea27-49aa-b4e7-47951cb06d81 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:37.265187 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7b1e00-ea27-49aa-b4e7-47951cb06d81 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:37.265625 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7b1e00-ea27-49aa-b4e7-47951cb06d81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:37.268826 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7b1e00-ea27-49aa-b4e7-47951cb06d81 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:37.287427 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 101/197] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:11:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:37.289493 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b1e00-ea27-49aa-b4e7-47951cb06d81 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:37.364837 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709ada31-8ddc-4c1c-8567-fb443957cd56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:37.367694 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709ada31-8ddc-4c1c-8567-fb443957cd56 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:37.486773 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b1e00-ea27-49aa-b4e7-47951cb06d81 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:37.493897 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:37.530377 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c502b42-1b0f-49a5-8286-f627a4b1c11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:37.531531 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c502b42-1b0f-49a5-8286-f627a4b1c11b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:37.531964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c502b42-1b0f-49a5-8286-f627a4b1c11b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:37.532148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 97/198] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:11:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:11:37.532565 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c502b42-1b0f-49a5-8286-f627a4b1c11b None admin] PATH_INFO: `/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:37.537696 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709ada31-8ddc-4c1c-8567-fb443957cd56 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:37.539711 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c502b42-1b0f-49a5-8286-f627a4b1c11b None admin] RBAC: Authorizing `identity:get_project(project_id=48caa0e9d7fe40d28e99abd1a7a6896f)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:37.541666 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c502b42-1b0f-49a5-8286-f627a4b1c11b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:37.543883 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 102/199] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:11:37 2022] GET /identity/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f => generated 298 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:37.545055 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c502b42-1b0f-49a5-8286-f627a4b1c11b None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:37.580201 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:37.663004 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709ada31-8ddc-4c1c-8567-fb443957cd56 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:37.674399 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c502b42-1b0f-49a5-8286-f627a4b1c11b None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:37.708395 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303bd934-db22-4ef9-8c60-8c83df4041be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:37.709408 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303bd934-db22-4ef9-8c60-8c83df4041be None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:37.709865 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303bd934-db22-4ef9-8c60-8c83df4041be None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:37.710102 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303bd934-db22-4ef9-8c60-8c83df4041be None admin] PATH_INFO: `/v3/projects/ee325f0b1b08455686f9e0a696ca57c8` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:37.711753 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303bd934-db22-4ef9-8c60-8c83df4041be None admin] RBAC: Authorizing `identity:get_project(project_id=ee325f0b1b08455686f9e0a696ca57c8)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:37.713603 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303bd934-db22-4ef9-8c60-8c83df4041be None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:37.716028 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 98/200] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:11:37 2022] GET /identity/v3/projects/ee325f0b1b08455686f9e0a696ca57c8 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:37.717298 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303bd934-db22-4ef9-8c60-8c83df4041be None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:37.723023 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a317aab-a6a5-4cc9-a935-d2cf499b695f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:37.724490 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a317aab-a6a5-4cc9-a935-d2cf499b695f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:37.725095 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a317aab-a6a5-4cc9-a935-d2cf499b695f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:37.725723 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a317aab-a6a5-4cc9-a935-d2cf499b695f None admin] PATH_INFO: `/v3/users/cd79af98206947fdb9a546a7cb70e1a6` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:37.727346 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a317aab-a6a5-4cc9-a935-d2cf499b695f None admin] RBAC: Authorizing `identity:get_user(user_id=cd79af98206947fdb9a546a7cb70e1a6)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:37.729216 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a317aab-a6a5-4cc9-a935-d2cf499b695f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:37.731581 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 103/201] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:11:37 2022] GET /identity/v3/users/cd79af98206947fdb9a546a7cb70e1a6 => generated 283 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:37.732765 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a317aab-a6a5-4cc9-a935-d2cf499b695f None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:37.770551 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:37.842951 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303bd934-db22-4ef9-8c60-8c83df4041be None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:37.857341 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:37.870369 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a317aab-a6a5-4cc9-a935-d2cf499b695f None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:37.887761 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff6cab-e9e1-4076-b504-48f4d3b8181d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:37.889087 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff6cab-e9e1-4076-b504-48f4d3b8181d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:37.889087 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff6cab-e9e1-4076-b504-48f4d3b8181d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:37.889362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff6cab-e9e1-4076-b504-48f4d3b8181d None admin] PATH_INFO: `/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:37.890675 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff6cab-e9e1-4076-b504-48f4d3b8181d None admin] RBAC: Authorizing `identity:get_project(project_id=48caa0e9d7fe40d28e99abd1a7a6896f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:37.892185 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff6cab-e9e1-4076-b504-48f4d3b8181d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:37.894512 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 99/202] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:11:37 2022] GET /identity/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:37.900503 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff6cab-e9e1-4076-b504-48f4d3b8181d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:37.924950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b2feb2-eb1f-449a-8e48-2192487fbf50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:37.926461 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2feb2-eb1f-449a-8e48-2192487fbf50 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:37.927181 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2feb2-eb1f-449a-8e48-2192487fbf50 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:37.928172 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2feb2-eb1f-449a-8e48-2192487fbf50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:37.929874 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b2feb2-eb1f-449a-8e48-2192487fbf50 None admin] RBAC: Adding query filter params (role.id=46e77bb17fc3465496a7a3a29d210042, scope.project.id=48caa0e9d7fe40d28e99abd1a7a6896f, user.id=786bcdb4dc274deb8280030ff16c6d0a) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:37.930658 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b2feb2-eb1f-449a-8e48-2192487fbf50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:37.933280 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b2feb2-eb1f-449a-8e48-2192487fbf50 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:37.943229 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 104/203] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:11:37 2022] GET /identity/v3/role_assignments?user.id=786bcdb4dc274deb8280030ff16c6d0a&scope.project.id=48caa0e9d7fe40d28e99abd1a7a6896f&role.id=46e77bb17fc3465496a7a3a29d210042 => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:37.944526 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b2feb2-eb1f-449a-8e48-2192487fbf50 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:38.020244 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff6cab-e9e1-4076-b504-48f4d3b8181d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:38.034392 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:38.074460 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d50a3f3-908f-4bef-8ec8-54fbf4ebb685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:38.074845 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50a3f3-908f-4bef-8ec8-54fbf4ebb685 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:38.075058 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50a3f3-908f-4bef-8ec8-54fbf4ebb685 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:38.075493 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50a3f3-908f-4bef-8ec8-54fbf4ebb685 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:38.076429 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d50a3f3-908f-4bef-8ec8-54fbf4ebb685 None admin] RBAC: Adding query filter params (role.id=98f0943a610f4a9e9f8de54f6e9f717d, scope.project.id=ee325f0b1b08455686f9e0a696ca57c8, user.id=cd79af98206947fdb9a546a7cb70e1a6) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:38.076745 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d50a3f3-908f-4bef-8ec8-54fbf4ebb685 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:38.078720 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d50a3f3-908f-4bef-8ec8-54fbf4ebb685 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:38.085277 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 100/204] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:11:37 2022] GET /identity/v3/role_assignments?user.id=cd79af98206947fdb9a546a7cb70e1a6&scope.project.id=ee325f0b1b08455686f9e0a696ca57c8&role.id=98f0943a610f4a9e9f8de54f6e9f717d => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:38.086415 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d50a3f3-908f-4bef-8ec8-54fbf4ebb685 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:38.121041 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b2feb2-eb1f-449a-8e48-2192487fbf50 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:38.182701 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26925d3d-26cc-497c-873c-bca957909c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:38.184762 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26925d3d-26cc-497c-873c-bca957909c94 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:38.185525 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26925d3d-26cc-497c-873c-bca957909c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:38.187066 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26925d3d-26cc-497c-873c-bca957909c94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:38.244322 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:38.259168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d50a3f3-908f-4bef-8ec8-54fbf4ebb685 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:38.310435 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e98473e-1685-4dfa-8e37-72db12dc17a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:38.311838 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98473e-1685-4dfa-8e37-72db12dc17a5 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:38.312302 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98473e-1685-4dfa-8e37-72db12dc17a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:38.313356 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98473e-1685-4dfa-8e37-72db12dc17a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:38.314484 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e98473e-1685-4dfa-8e37-72db12dc17a5 None admin] RBAC: Adding query filter params (role.id=5efb4931668c4edf87773a1f5d47f005, scope.project.id=48caa0e9d7fe40d28e99abd1a7a6896f, user.id=2ed789dda2154c888945486c0b8c55d3) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:38.315284 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e98473e-1685-4dfa-8e37-72db12dc17a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:38.317547 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e98473e-1685-4dfa-8e37-72db12dc17a5 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:38.326415 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 101/205] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:11:38 2022] GET /identity/v3/role_assignments?user.id=2ed789dda2154c888945486c0b8c55d3&scope.project.id=48caa0e9d7fe40d28e99abd1a7a6896f&role.id=5efb4931668c4edf87773a1f5d47f005 => generated 608 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:38.337987 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26925d3d-26cc-497c-873c-bca957909c94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:38.341052 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26925d3d-26cc-497c-873c-bca957909c94 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:38.586115 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 105/206] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:11:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 642 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:11:38.590920 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e98473e-1685-4dfa-8e37-72db12dc17a5 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:38.666774 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:38.759188 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e98473e-1685-4dfa-8e37-72db12dc17a5 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:38.799667 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d8e0b7-3bd9-460f-a1b5-f8da09325cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:38.800644 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d8e0b7-3bd9-460f-a1b5-f8da09325cdc None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:38.800937 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d8e0b7-3bd9-460f-a1b5-f8da09325cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:38.801218 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d8e0b7-3bd9-460f-a1b5-f8da09325cdc None admin] PATH_INFO: `/v3/projects/ee325f0b1b08455686f9e0a696ca57c8` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:38.802725 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d8e0b7-3bd9-460f-a1b5-f8da09325cdc None admin] RBAC: Authorizing `identity:get_project(project_id=ee325f0b1b08455686f9e0a696ca57c8)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:38.804345 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d8e0b7-3bd9-460f-a1b5-f8da09325cdc None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:38.806201 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 102/207] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:11:38 2022] GET /identity/v3/projects/ee325f0b1b08455686f9e0a696ca57c8 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:38.819438 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26925d3d-26cc-497c-873c-bca957909c94 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:38.901441 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:38.960941 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26925d3d-26cc-497c-873c-bca957909c94 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:39.038656 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4450e5d-db0f-4fc0-87b9-6d90171a48d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:39.039851 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4450e5d-db0f-4fc0-87b9-6d90171a48d9 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:39.040341 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4450e5d-db0f-4fc0-87b9-6d90171a48d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:39.040848 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4450e5d-db0f-4fc0-87b9-6d90171a48d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:39.042542 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4450e5d-db0f-4fc0-87b9-6d90171a48d9 None admin] RBAC: Adding query filter params (role.id=46e77bb17fc3465496a7a3a29d210042, scope.project.id=ee325f0b1b08455686f9e0a696ca57c8, user.id=cd79af98206947fdb9a546a7cb70e1a6) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:39.043084 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4450e5d-db0f-4fc0-87b9-6d90171a48d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:39.046105 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4450e5d-db0f-4fc0-87b9-6d90171a48d9 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:39.058404 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 106/208] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:11:38 2022] GET /identity/v3/role_assignments?user.id=cd79af98206947fdb9a546a7cb70e1a6&scope.project.id=ee325f0b1b08455686f9e0a696ca57c8&role.id=46e77bb17fc3465496a7a3a29d210042 => generated 259 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:40.204992 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d5912-6484-41f2-9483-03d607c8ab94 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:40.205549 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d5912-6484-41f2-9483-03d607c8ab94 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:40.205549 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d5912-6484-41f2-9483-03d607c8ab94 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:40.207877 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 103/209] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:11:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:11:40.214147 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b7f887-4253-45f7-9ce9-6c54c9a620f9 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:40.215141 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b7f887-4253-45f7-9ce9-6c54c9a620f9 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:40.215645 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b7f887-4253-45f7-9ce9-6c54c9a620f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:40.235589 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-38b7f887-4253-45f7-9ce9-6c54c9a620f9 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:11:40.237332 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-38b7f887-4253-45f7-9ce9-6c54c9a620f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:11:40.322315 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 107/210] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:11:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:11:40.352228 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d5912-6484-41f2-9483-03d607c8ab94 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:40.483009 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d5912-6484-41f2-9483-03d607c8ab94 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:40.486676 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:40.529936 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cda4b33-b979-4f1e-af44-8e66c48327cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:40.531151 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda4b33-b979-4f1e-af44-8e66c48327cf None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:40.531665 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda4b33-b979-4f1e-af44-8e66c48327cf None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:40.532211 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda4b33-b979-4f1e-af44-8e66c48327cf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:40.533825 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cda4b33-b979-4f1e-af44-8e66c48327cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:40.535732 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cda4b33-b979-4f1e-af44-8e66c48327cf None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:40.538117 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 104/211] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:11:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:40.544512 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b7f887-4253-45f7-9ce9-6c54c9a620f9 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:40.622648 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:40.674550 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b7f887-4253-45f7-9ce9-6c54c9a620f9 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:40.716940 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b344e9a9-e013-42c9-a91d-79ca5dba392e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:40.718096 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344e9a9-e013-42c9-a91d-79ca5dba392e None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:40.718579 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344e9a9-e013-42c9-a91d-79ca5dba392e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:40.719069 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344e9a9-e013-42c9-a91d-79ca5dba392e None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:40.720019 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b344e9a9-e013-42c9-a91d-79ca5dba392e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:40.721735 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b344e9a9-e013-42c9-a91d-79ca5dba392e None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:40.752433 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 108/212] 10.222.0.87 () {60 vars in 1132 bytes} [Tue Jun 21 18:11:40 2022] POST /identity/v3/users => generated 297 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:11:40.835107 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db156d23-0646-4a6c-9632-b8ba04add3b2 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:40.835756 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db156d23-0646-4a6c-9632-b8ba04add3b2 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:40.836249 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db156d23-0646-4a6c-9632-b8ba04add3b2 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:40.838064 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 105/213] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:11:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:11:40.847602 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88305987-9314-4dc1-8e97-90f0793d18b3 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:40.848457 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88305987-9314-4dc1-8e97-90f0793d18b3 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:40.849179 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88305987-9314-4dc1-8e97-90f0793d18b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:40.879897 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2740301d-f513-46dc-a4bb-06505cedbcfb None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:40.880136 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-88305987-9314-4dc1-8e97-90f0793d18b3 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:11:40.880562 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2740301d-f513-46dc-a4bb-06505cedbcfb None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:40.880985 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2740301d-f513-46dc-a4bb-06505cedbcfb None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:40.882252 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-88305987-9314-4dc1-8e97-90f0793d18b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:11:40.882967 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 106/214] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:11:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:11:40.889443 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea86af1-2d61-4bb6-a91c-8fe6acd56efc None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:40.890148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea86af1-2d61-4bb6-a91c-8fe6acd56efc None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:40.890796 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea86af1-2d61-4bb6-a91c-8fe6acd56efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:40.911857 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3ea86af1-2d61-4bb6-a91c-8fe6acd56efc None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:11:40.913895 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3ea86af1-2d61-4bb6-a91c-8fe6acd56efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:11:41.002177 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 109/215] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:11:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:11:41.040141 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88305987-9314-4dc1-8e97-90f0793d18b3 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:41.046819 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 107/216] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:11:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:11:41.086888 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea86af1-2d61-4bb6-a91c-8fe6acd56efc None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:41.095838 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:41.219857 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88305987-9314-4dc1-8e97-90f0793d18b3 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:41.270805 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891668a8-b2c7-4515-b8b5-f146c466e5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:41.272460 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891668a8-b2c7-4515-b8b5-f146c466e5b2 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:41.272923 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891668a8-b2c7-4515-b8b5-f146c466e5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:41.273330 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891668a8-b2c7-4515-b8b5-f146c466e5b2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:41.293990 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891668a8-b2c7-4515-b8b5-f146c466e5b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:11:41.300395 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea86af1-2d61-4bb6-a91c-8fe6acd56efc None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:41.304148 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-891668a8-b2c7-4515-b8b5-f146c466e5b2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:11:41.304148 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:11:41.304148 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:11:41.304148 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:11:41.304148 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:11:41.304148 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:11:41.304148 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:11:41.304148 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:11:41.304148 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:11:41.304148 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:11:41.304148 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:11:41.304148 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:11:41.304148 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:11:41.304148 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:11:41.304148 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:11:41.304148 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:11:41.305200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:11:41.305200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:11:41.305200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:11:41.305200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:11:41.305200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:11:41.305200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:11:41.305200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:11:41.305200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:11:41.305200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:11:41.305200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:11:41.305200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:11:41.305200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:11:41.305200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:11:41.305200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:11:41.305200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:11:41.305200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:11:41.306173 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:11:41.306173 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:11:41.306173 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:11:41.306173 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:11:41.306173 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:11:41.306173 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:11:41.306173 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:11:41.306173 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:11:41.306173 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:11:41.306173 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:11:41.306173 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:11:41.306173 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:11:41.307859 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:41.309993 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 110/217] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:11:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:11:41.323326 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891668a8-b2c7-4515-b8b5-f146c466e5b2 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:41.389590 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abbf409-90af-42ee-9d37-def8a8516417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:41.392183 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abbf409-90af-42ee-9d37-def8a8516417 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:41.392858 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abbf409-90af-42ee-9d37-def8a8516417 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:41.393860 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abbf409-90af-42ee-9d37-def8a8516417 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:41.420210 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abbf409-90af-42ee-9d37-def8a8516417 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:11:41.439475 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-1abbf409-90af-42ee-9d37-def8a8516417 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:11:41.439475 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:11:41.439475 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:11:41.439475 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:11:41.439475 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:11:41.439475 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:11:41.439475 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:11:41.439475 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:11:41.439475 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:11:41.439475 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:11:41.439475 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:11:41.439475 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:11:41.439475 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:11:41.439475 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:11:41.439475 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:11:41.439475 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:11:41.440607 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:11:41.440607 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:11:41.440607 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:11:41.440607 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:11:41.440607 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:11:41.440607 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:11:41.440607 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:11:41.440607 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:11:41.440607 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:11:41.440607 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:11:41.440607 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:11:41.440607 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:11:41.440607 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:11:41.440607 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:11:41.440607 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:11:41.440607 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:11:41.442147 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:11:41.442147 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:11:41.442147 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:11:41.442147 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:11:41.442147 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:11:41.442147 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:11:41.442147 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:11:41.442147 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:11:41.442147 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:11:41.442147 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:11:41.442147 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:11:41.442147 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:11:41.443095 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 108/218] 10.222.0.87 () {56 vars in 1096 bytes} [Tue Jun 21 18:11:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:11:41.455120 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abbf409-90af-42ee-9d37-def8a8516417 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:41.481145 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:41.499709 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891668a8-b2c7-4515-b8b5-f146c466e5b2 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:41.596946 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abbf409-90af-42ee-9d37-def8a8516417 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:41.621998 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e07d29-215f-47df-b490-8a664f6ce15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:41.623080 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e07d29-215f-47df-b490-8a664f6ce15e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:41.623569 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e07d29-215f-47df-b490-8a664f6ce15e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:41.624076 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e07d29-215f-47df-b490-8a664f6ce15e None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:41.625064 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e07d29-215f-47df-b490-8a664f6ce15e None admin] RBAC: Adding query filter params (name=member) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:41.625591 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e07d29-215f-47df-b490-8a664f6ce15e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:41.627269 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e07d29-215f-47df-b490-8a664f6ce15e None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:41.649877 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 111/219] 10.222.0.87 () {56 vars in 1101 bytes} [Tue Jun 21 18:11:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:11:41.658364 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e07d29-215f-47df-b490-8a664f6ce15e None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:41.685580 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3141eca-398b-4d9a-80af-77053d439705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:41.686767 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3141eca-398b-4d9a-80af-77053d439705 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:41.687265 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3141eca-398b-4d9a-80af-77053d439705 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:41.687752 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3141eca-398b-4d9a-80af-77053d439705 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:41.688801 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3141eca-398b-4d9a-80af-77053d439705 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:41.689563 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3141eca-398b-4d9a-80af-77053d439705 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:41.692414 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3141eca-398b-4d9a-80af-77053d439705 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:41.712819 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 109/220] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:11:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:11:41.722962 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3141eca-398b-4d9a-80af-77053d439705 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:41.789385 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:41.853601 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e07d29-215f-47df-b490-8a664f6ce15e None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:41.932648 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6f0537-d45c-4815-9e5e-5524f1a59f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:41.933557 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f0537-d45c-4815-9e5e-5524f1a59f13 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:41.933881 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f0537-d45c-4815-9e5e-5524f1a59f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:41.934166 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f0537-d45c-4815-9e5e-5524f1a59f13 None admin] PATH_INFO: `/v3/users/99487f792aae445d847c795ec9018376` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:41.970388 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6f0537-d45c-4815-9e5e-5524f1a59f13 None admin] RBAC: Authorizing `identity:get_user(user_id=99487f792aae445d847c795ec9018376)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:41.971977 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6f0537-d45c-4815-9e5e-5524f1a59f13 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:41.973946 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 112/221] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:11:41 2022] GET /identity/v3/users/99487f792aae445d847c795ec9018376 => generated 297 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:41.980111 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6f0537-d45c-4815-9e5e-5524f1a59f13 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:41.982646 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3141eca-398b-4d9a-80af-77053d439705 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:42.003246 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:42.073831 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd28143d-4cdd-4f55-ab86-586a782431d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:42.074734 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28143d-4cdd-4f55-ab86-586a782431d0 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:42.075019 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28143d-4cdd-4f55-ab86-586a782431d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:42.075295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28143d-4cdd-4f55-ab86-586a782431d0 None admin] PATH_INFO: `/v3/users/cd79af98206947fdb9a546a7cb70e1a6` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:42.077801 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd28143d-4cdd-4f55-ab86-586a782431d0 None admin] RBAC: Authorizing `identity:get_user(user_id=cd79af98206947fdb9a546a7cb70e1a6)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:42.079302 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd28143d-4cdd-4f55-ab86-586a782431d0 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:42.083650 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 110/222] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:11:41 2022] GET /identity/v3/users/cd79af98206947fdb9a546a7cb70e1a6 => generated 283 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:42.090762 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd28143d-4cdd-4f55-ab86-586a782431d0 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:42.140120 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6f0537-d45c-4815-9e5e-5524f1a59f13 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:42.181532 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2c8e58-a16c-4ac2-9deb-f3568ea594c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:42.182255 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c8e58-a16c-4ac2-9deb-f3568ea594c6 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:42.182544 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c8e58-a16c-4ac2-9deb-f3568ea594c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:42.182827 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c8e58-a16c-4ac2-9deb-f3568ea594c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:42.279870 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2c8e58-a16c-4ac2-9deb-f3568ea594c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:42.282769 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2c8e58-a16c-4ac2-9deb-f3568ea594c6 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:42.324413 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:42.485873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 113/223] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:11:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:11:42.489363 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2c8e58-a16c-4ac2-9deb-f3568ea594c6 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:42.503692 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:42.540944 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd28143d-4cdd-4f55-ab86-586a782431d0 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:42.624889 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add2aea8-c70f-451b-9624-d4c7b30aa500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:42.625841 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2aea8-c70f-451b-9624-d4c7b30aa500 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:42.626230 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2aea8-c70f-451b-9624-d4c7b30aa500 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:42.626518 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2aea8-c70f-451b-9624-d4c7b30aa500 None admin] PATH_INFO: `/v3/projects/ee325f0b1b08455686f9e0a696ca57c8` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:42.627968 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add2aea8-c70f-451b-9624-d4c7b30aa500 None admin] RBAC: Authorizing `identity:get_project(project_id=ee325f0b1b08455686f9e0a696ca57c8)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:42.629919 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add2aea8-c70f-451b-9624-d4c7b30aa500 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:42.630496 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2c8e58-a16c-4ac2-9deb-f3568ea594c6 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:42.631790 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 111/224] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:11:42 2022] GET /identity/v3/projects/ee325f0b1b08455686f9e0a696ca57c8 => generated 302 bytes in 542 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:42.633162 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5051477b-8efb-4d6d-88f7-c50669384ea6 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:42.633457 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5051477b-8efb-4d6d-88f7-c50669384ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:42.633770 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5051477b-8efb-4d6d-88f7-c50669384ea6 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:42.635196 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 112/225] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:11:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:11:42.642770 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de17e8b-d4cb-41ed-9640-f5fd5d61a715 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:42.642960 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de17e8b-d4cb-41ed-9640-f5fd5d61a715 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:42.643242 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de17e8b-d4cb-41ed-9640-f5fd5d61a715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:42.657625 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:42.664343 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-8de17e8b-d4cb-41ed-9640-f5fd5d61a715 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:11:42.665626 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8de17e8b-d4cb-41ed-9640-f5fd5d61a715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:11:42.684237 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d9bf2-f404-4253-b14e-69141f5ac1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:42.685262 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d9bf2-f404-4253-b14e-69141f5ac1bf None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:42.685612 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d9bf2-f404-4253-b14e-69141f5ac1bf None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:42.686000 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d9bf2-f404-4253-b14e-69141f5ac1bf None admin] PATH_INFO: `/v3/projects/ee325f0b1b08455686f9e0a696ca57c8` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:42.687419 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355d9bf2-f404-4253-b14e-69141f5ac1bf None admin] RBAC: Authorizing `identity:get_project(project_id=ee325f0b1b08455686f9e0a696ca57c8)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:42.689080 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355d9bf2-f404-4253-b14e-69141f5ac1bf None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:42.696969 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 114/226] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:11:42 2022] GET /identity/v3/projects/ee325f0b1b08455686f9e0a696ca57c8 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:42.698106 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d9bf2-f404-4253-b14e-69141f5ac1bf None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:42.776414 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 113/227] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:11:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:11:42.777809 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de17e8b-d4cb-41ed-9640-f5fd5d61a715 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:42.873017 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d9bf2-f404-4253-b14e-69141f5ac1bf None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:42.905582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:42.921668 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de17e8b-d4cb-41ed-9640-f5fd5d61a715 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:42.940083 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dcef70-5a81-4058-8455-287380e05369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:42.941606 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dcef70-5a81-4058-8455-287380e05369 None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:42.942191 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dcef70-5a81-4058-8455-287380e05369 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:42.942601 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dcef70-5a81-4058-8455-287380e05369 None admin] PATH_INFO: `/v3/projects/ee325f0b1b08455686f9e0a696ca57c8/users/cd79af98206947fdb9a546a7cb70e1a6/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:42.946617 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dcef70-5a81-4058-8455-287380e05369 None admin] RBAC: Authorizing `identity:create_grant(project_id=ee325f0b1b08455686f9e0a696ca57c8, user_id=cd79af98206947fdb9a546a7cb70e1a6, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:42.948655 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dcef70-5a81-4058-8455-287380e05369 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:42.965916 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 115/228] 10.222.0.87 () {58 vars in 1439 bytes} [Tue Jun 21 18:11:42 2022] PUT /identity/v3/projects/ee325f0b1b08455686f9e0a696ca57c8/users/cd79af98206947fdb9a546a7cb70e1a6/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:11:42.972458 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dcef70-5a81-4058-8455-287380e05369 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:42.982296 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b51478-bfde-46bf-a500-63949e5dccb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:42.983395 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b51478-bfde-46bf-a500-63949e5dccb7 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:42.984478 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b51478-bfde-46bf-a500-63949e5dccb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:42.985079 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b51478-bfde-46bf-a500-63949e5dccb7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:42.986417 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b51478-bfde-46bf-a500-63949e5dccb7 None admin] RBAC: Adding query filter params (role.id=da6d4a0ab8c04809ba120170fb4ae8a9, scope.project.id=ee325f0b1b08455686f9e0a696ca57c8, user.id=99487f792aae445d847c795ec9018376) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:42.988655 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b51478-bfde-46bf-a500-63949e5dccb7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:42.990506 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b51478-bfde-46bf-a500-63949e5dccb7 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:43.004882 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 114/229] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:11:42 2022] GET /identity/v3/role_assignments?user.id=99487f792aae445d847c795ec9018376&scope.project.id=ee325f0b1b08455686f9e0a696ca57c8&role.id=da6d4a0ab8c04809ba120170fb4ae8a9 => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:43.054253 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:43.152318 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dcef70-5a81-4058-8455-287380e05369 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:43.228458 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3268d-4ff2-4739-9360-0a1aeb4b8e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:43.230134 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3268d-4ff2-4739-9360-0a1aeb4b8e8e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:43.230791 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3268d-4ff2-4739-9360-0a1aeb4b8e8e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:43.231474 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3268d-4ff2-4739-9360-0a1aeb4b8e8e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:43.257442 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf3268d-4ff2-4739-9360-0a1aeb4b8e8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:11:43.266360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-cdf3268d-4ff2-4739-9360-0a1aeb4b8e8e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:11:43.266360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:11:43.266360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:11:43.266360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:11:43.266360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:11:43.266360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:11:43.266360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:11:43.266360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:11:43.266360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:11:43.266360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:11:43.266360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:11:43.266360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:11:43.266360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:11:43.266360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:11:43.266360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:11:43.266360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:11:43.267973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:11:43.267973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:11:43.267973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:11:43.267973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:11:43.267973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:11:43.267973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:11:43.267973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:11:43.267973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:11:43.267973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:11:43.267973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:11:43.267973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:11:43.267973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:11:43.267973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:11:43.267973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:11:43.267973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:11:43.267973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:11:43.269046 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:11:43.269046 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:11:43.269046 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:11:43.269046 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:11:43.269046 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:11:43.269046 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:11:43.269046 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:11:43.269046 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:11:43.269046 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:11:43.269046 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:11:43.269046 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:11:43.269046 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:11:43.272179 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 116/230] 10.222.0.87 () {56 vars in 1114 bytes} [Tue Jun 21 18:11:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:11:43.278480 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b51478-bfde-46bf-a500-63949e5dccb7 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:43.415835 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b51478-bfde-46bf-a500-63949e5dccb7 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:43.467291 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0845fa45-8e5d-4cae-abdd-d6b5398d595a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:43.468418 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845fa45-8e5d-4cae-abdd-d6b5398d595a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:43.468903 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845fa45-8e5d-4cae-abdd-d6b5398d595a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:43.469373 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845fa45-8e5d-4cae-abdd-d6b5398d595a None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:43.470427 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0845fa45-8e5d-4cae-abdd-d6b5398d595a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:43.470893 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0845fa45-8e5d-4cae-abdd-d6b5398d595a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:43.472542 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0845fa45-8e5d-4cae-abdd-d6b5398d595a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:43.486730 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 115/231] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:11:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:43.495557 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3268d-4ff2-4739-9360-0a1aeb4b8e8e None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:43.656056 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3268d-4ff2-4739-9360-0a1aeb4b8e8e None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:43.659256 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:43.720543 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc2b31f-7f28-4c52-9dfe-9019a489d438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:43.722202 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc2b31f-7f28-4c52-9dfe-9019a489d438 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:43.722706 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc2b31f-7f28-4c52-9dfe-9019a489d438 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:43.723199 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc2b31f-7f28-4c52-9dfe-9019a489d438 None admin] PATH_INFO: `/v3/users/cd79af98206947fdb9a546a7cb70e1a6` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:43.725541 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc2b31f-7f28-4c52-9dfe-9019a489d438 None admin] RBAC: Authorizing `identity:get_user(user_id=cd79af98206947fdb9a546a7cb70e1a6)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:43.728373 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc2b31f-7f28-4c52-9dfe-9019a489d438 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:43.731852 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 117/232] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:11:43 2022] GET /identity/v3/users/cd79af98206947fdb9a546a7cb70e1a6 => generated 283 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:43.741215 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0845fa45-8e5d-4cae-abdd-d6b5398d595a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:43.772905 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:43.822247 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c4e4c-c6c2-4229-9799-f7e0aa97f2ba None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:43.822579 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c4e4c-c6c2-4229-9799-f7e0aa97f2ba None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:43.822800 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c4e4c-c6c2-4229-9799-f7e0aa97f2ba None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:43.824277 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 118/233] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:11:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:11:43.830295 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1abe069-6b3e-43a5-9870-62edae90bb96 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:43.830703 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1abe069-6b3e-43a5-9870-62edae90bb96 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:43.831084 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1abe069-6b3e-43a5-9870-62edae90bb96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:43.857597 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-e1abe069-6b3e-43a5-9870-62edae90bb96 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:11:43.859374 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e1abe069-6b3e-43a5-9870-62edae90bb96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:11:43.997320 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0845fa45-8e5d-4cae-abdd-d6b5398d595a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:44.051198 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 119/234] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:11:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:11:44.070981 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2924f2-f787-4bc1-9d0a-2bd722c68c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:44.074037 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2924f2-f787-4bc1-9d0a-2bd722c68c32 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:44.074776 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2924f2-f787-4bc1-9d0a-2bd722c68c32 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:44.075517 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2924f2-f787-4bc1-9d0a-2bd722c68c32 None admin] PATH_INFO: `/v3/projects/ee325f0b1b08455686f9e0a696ca57c8` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:44.077980 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2924f2-f787-4bc1-9d0a-2bd722c68c32 None admin] RBAC: Authorizing `identity:get_project(project_id=ee325f0b1b08455686f9e0a696ca57c8)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:44.080945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2924f2-f787-4bc1-9d0a-2bd722c68c32 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:44.084559 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 116/235] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:11:43 2022] GET /identity/v3/projects/ee325f0b1b08455686f9e0a696ca57c8 => generated 302 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:44.091145 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1abe069-6b3e-43a5-9870-62edae90bb96 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:44.093939 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2924f2-f787-4bc1-9d0a-2bd722c68c32 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:44.308144 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:44.322256 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1abe069-6b3e-43a5-9870-62edae90bb96 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:44.324525 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:44.351462 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2924f2-f787-4bc1-9d0a-2bd722c68c32 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:44.401511 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b71d9d3-8a99-43d4-a8bd-6bc25b40d4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:44.402511 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b71d9d3-8a99-43d4-a8bd-6bc25b40d4f2 None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:44.402799 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b71d9d3-8a99-43d4-a8bd-6bc25b40d4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:44.403078 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b71d9d3-8a99-43d4-a8bd-6bc25b40d4f2 None admin] PATH_INFO: `/v3/projects/ee325f0b1b08455686f9e0a696ca57c8/users/cd79af98206947fdb9a546a7cb70e1a6/roles/46e77bb17fc3465496a7a3a29d210042` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:44.406587 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b71d9d3-8a99-43d4-a8bd-6bc25b40d4f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=ee325f0b1b08455686f9e0a696ca57c8, user_id=cd79af98206947fdb9a546a7cb70e1a6, role_id=46e77bb17fc3465496a7a3a29d210042)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:44.408291 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b71d9d3-8a99-43d4-a8bd-6bc25b40d4f2 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:44.421117 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c803e-6393-46d3-b9f6-4248456885ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:44.422689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c803e-6393-46d3-b9f6-4248456885ee None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:44.423896 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c803e-6393-46d3-b9f6-4248456885ee None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:44.424560 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c803e-6393-46d3-b9f6-4248456885ee None admin] PATH_INFO: `/v3/domains/default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:44.426531 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55c803e-6393-46d3-b9f6-4248456885ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:44.428277 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 120/236] 10.222.0.87 () {58 vars in 1439 bytes} [Tue Jun 21 18:11:44 2022] PUT /identity/v3/projects/ee325f0b1b08455686f9e0a696ca57c8/users/cd79af98206947fdb9a546a7cb70e1a6/roles/46e77bb17fc3465496a7a3a29d210042 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:11:44.428739 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55c803e-6393-46d3-b9f6-4248456885ee None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:44.433758 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ebd33-2015-4f0e-9ed0-f7385f0b48e9 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:44.433758 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ebd33-2015-4f0e-9ed0-f7385f0b48e9 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:44.433758 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ebd33-2015-4f0e-9ed0-f7385f0b48e9 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:44.433758 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 121/237] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:11:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:11:44.439901 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 117/238] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:11:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:44.441356 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7af05e-e488-4089-a8b1-9e662475912d None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:44.441661 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7af05e-e488-4089-a8b1-9e662475912d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:44.442208 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7af05e-e488-4089-a8b1-9e662475912d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:44.447606 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c803e-6393-46d3-b9f6-4248456885ee None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:44.486515 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-8f7af05e-e488-4089-a8b1-9e662475912d None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:11:44.489355 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8f7af05e-e488-4089-a8b1-9e662475912d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:11:44.604084 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 122/239] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:11:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:11:44.641562 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7af05e-e488-4089-a8b1-9e662475912d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:44.768975 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c803e-6393-46d3-b9f6-4248456885ee None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:44.898430 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:44.919734 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db6571-7375-4f62-9db9-4610d3e70407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:44.921549 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db6571-7375-4f62-9db9-4610d3e70407 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:44.922510 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db6571-7375-4f62-9db9-4610d3e70407 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:44.923387 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db6571-7375-4f62-9db9-4610d3e70407 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:44.925344 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30db6571-7375-4f62-9db9-4610d3e70407 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:44.927580 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7af05e-e488-4089-a8b1-9e662475912d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:44.928816 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30db6571-7375-4f62-9db9-4610d3e70407 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:44.972673 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 118/240] 10.222.0.87 () {60 vars in 1132 bytes} [Tue Jun 21 18:11:44 2022] POST /identity/v3/users => generated 293 bytes in 526 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:11:45.053784 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fced12-01d3-4828-8d04-18af0926b085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:45.055932 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fced12-01d3-4828-8d04-18af0926b085 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:45.056938 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fced12-01d3-4828-8d04-18af0926b085 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:45.057865 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fced12-01d3-4828-8d04-18af0926b085 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:45.080294 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fced12-01d3-4828-8d04-18af0926b085 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:11:45.089255 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-99fced12-01d3-4828-8d04-18af0926b085 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:11:45.089255 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:11:45.089255 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:11:45.089255 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:11:45.089255 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:11:45.089255 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:11:45.089255 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:11:45.089255 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:11:45.089255 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:11:45.089255 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:11:45.089255 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:11:45.089255 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:11:45.089255 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:11:45.089255 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:11:45.089255 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:11:45.089255 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:11:45.090873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:11:45.090873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:11:45.090873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:11:45.090873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:11:45.090873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:11:45.090873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:11:45.090873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:11:45.090873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:11:45.090873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:11:45.090873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:11:45.090873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:11:45.090873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:11:45.090873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:11:45.090873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:11:45.090873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:11:45.090873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:11:45.092993 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:11:45.092993 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:11:45.092993 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:11:45.092993 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:11:45.092993 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:11:45.092993 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:11:45.092993 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:11:45.092993 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:11:45.092993 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:11:45.092993 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:11:45.092993 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:11:45.092993 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:11:45.092993 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 123/241] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:11:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 451 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:11:45.097998 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db6571-7375-4f62-9db9-4610d3e70407 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:45.135957 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:45.364734 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db6571-7375-4f62-9db9-4610d3e70407 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:45.442753 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe17f6e-9eaa-4a84-b249-a902ba2aaf4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:45.443855 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe17f6e-9eaa-4a84-b249-a902ba2aaf4c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:45.444346 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe17f6e-9eaa-4a84-b249-a902ba2aaf4c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:45.444829 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe17f6e-9eaa-4a84-b249-a902ba2aaf4c None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:45.446277 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe17f6e-9eaa-4a84-b249-a902ba2aaf4c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:45.446826 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe17f6e-9eaa-4a84-b249-a902ba2aaf4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:45.449746 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe17f6e-9eaa-4a84-b249-a902ba2aaf4c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:45.469497 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 119/242] 10.222.0.87 () {56 vars in 1101 bytes} [Tue Jun 21 18:11:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:45.480348 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fced12-01d3-4828-8d04-18af0926b085 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:45.745083 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fced12-01d3-4828-8d04-18af0926b085 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:45.832297 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47013997-a1a4-4b8f-9dc6-e37bf2207f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:45.834164 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47013997-a1a4-4b8f-9dc6-e37bf2207f14 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:45.834723 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47013997-a1a4-4b8f-9dc6-e37bf2207f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:45.835230 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47013997-a1a4-4b8f-9dc6-e37bf2207f14 None admin] PATH_INFO: `/v3/users/ca6ab7a1cd61437288936f3c5c0f2494` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:45.873338 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47013997-a1a4-4b8f-9dc6-e37bf2207f14 None admin] RBAC: Authorizing `identity:get_user(user_id=ca6ab7a1cd61437288936f3c5c0f2494)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:45.875036 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47013997-a1a4-4b8f-9dc6-e37bf2207f14 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:45.877268 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 124/243] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:11:45 2022] GET /identity/v3/users/ca6ab7a1cd61437288936f3c5c0f2494 => generated 297 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:45.884656 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe17f6e-9eaa-4a84-b249-a902ba2aaf4c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:45.988633 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f64fd8-86d5-4f93-a66e-d7fd9d99f433 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:45.988633 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f64fd8-86d5-4f93-a66e-d7fd9d99f433 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:45.988633 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f64fd8-86d5-4f93-a66e-d7fd9d99f433 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:45.988633 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 125/244] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:11:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:11:45.997838 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:46.006550 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15c327-752b-4fdc-8048-3d9766a43882 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:46.007204 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15c327-752b-4fdc-8048-3d9766a43882 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:46.007769 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15c327-752b-4fdc-8048-3d9766a43882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:46.035465 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-4d15c327-752b-4fdc-8048-3d9766a43882 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:11:46.038786 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4d15c327-752b-4fdc-8048-3d9766a43882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:11:46.045943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:46.077360 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe17f6e-9eaa-4a84-b249-a902ba2aaf4c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:46.128262 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f43b2d5-de5d-47b9-8c17-9bb121cb5e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:46.129484 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f43b2d5-de5d-47b9-8c17-9bb121cb5e45 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:46.129973 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f43b2d5-de5d-47b9-8c17-9bb121cb5e45 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:46.130594 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f43b2d5-de5d-47b9-8c17-9bb121cb5e45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:46.233263 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f43b2d5-de5d-47b9-8c17-9bb121cb5e45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:46.235661 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f43b2d5-de5d-47b9-8c17-9bb121cb5e45 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:46.255837 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 126/245] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:11:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:11:46.328297 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d15c327-752b-4fdc-8048-3d9766a43882 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:46.382209 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:46.445394 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 120/246] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:11:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 21 18:11:46.454168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f43b2d5-de5d-47b9-8c17-9bb121cb5e45 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:46.625571 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d15c327-752b-4fdc-8048-3d9766a43882 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:46.689837 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b03369-f0e8-4a25-9776-028e7d88d64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:46.691509 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b03369-f0e8-4a25-9776-028e7d88d64a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:46.691509 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b03369-f0e8-4a25-9776-028e7d88d64a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:46.691957 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b03369-f0e8-4a25-9776-028e7d88d64a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:46.725408 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:46.729798 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b03369-f0e8-4a25-9776-028e7d88d64a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:11:46.744271 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-53b03369-f0e8-4a25-9776-028e7d88d64a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:11:46.744271 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:11:46.744271 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:11:46.744271 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:11:46.744271 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:11:46.744271 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:11:46.744271 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:11:46.744271 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:11:46.744271 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:11:46.744271 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:11:46.744271 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:11:46.744271 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:11:46.744271 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:11:46.744271 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:11:46.744271 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:11:46.744271 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:11:46.747865 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:11:46.747865 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:11:46.747865 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:11:46.747865 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:11:46.747865 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:11:46.747865 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:11:46.747865 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:11:46.747865 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:11:46.747865 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:11:46.747865 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:11:46.747865 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:11:46.747865 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:11:46.747865 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:11:46.747865 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:11:46.747865 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:11:46.747865 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:11:46.749446 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:11:46.749446 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:11:46.749446 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:11:46.749446 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:11:46.749446 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:11:46.749446 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:11:46.749446 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:11:46.749446 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:11:46.749446 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:11:46.749446 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:11:46.749446 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:11:46.749446 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:11:46.749446 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 127/247] 10.222.0.87 () {56 vars in 1096 bytes} [Tue Jun 21 18:11:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 418 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:11:46.752852 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b03369-f0e8-4a25-9776-028e7d88d64a None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:46.781997 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f43b2d5-de5d-47b9-8c17-9bb121cb5e45 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:46.867991 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9a87f4-9182-4f4e-9920-a7b2cb6292a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:46.868901 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a87f4-9182-4f4e-9920-a7b2cb6292a7 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:46.869360 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a87f4-9182-4f4e-9920-a7b2cb6292a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:46.869684 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a87f4-9182-4f4e-9920-a7b2cb6292a7 None admin] PATH_INFO: `/v3/projects/ee325f0b1b08455686f9e0a696ca57c8` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:46.875189 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9a87f4-9182-4f4e-9920-a7b2cb6292a7 None admin] RBAC: Authorizing `identity:get_project(project_id=ee325f0b1b08455686f9e0a696ca57c8)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:46.877690 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9a87f4-9182-4f4e-9920-a7b2cb6292a7 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:46.879894 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 121/248] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:11:46 2022] GET /identity/v3/projects/ee325f0b1b08455686f9e0a696ca57c8 => generated 302 bytes in 427 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:46.887203 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9a87f4-9182-4f4e-9920-a7b2cb6292a7 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:46.934677 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b03369-f0e8-4a25-9776-028e7d88d64a None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:46.973781 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:47.044688 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb95df78-0934-43c9-bd04-d0e1f25e1c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:47.045381 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95df78-0934-43c9-bd04-d0e1f25e1c9e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:47.045703 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95df78-0934-43c9-bd04-d0e1f25e1c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:47.045988 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95df78-0934-43c9-bd04-d0e1f25e1c9e None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:47.046782 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb95df78-0934-43c9-bd04-d0e1f25e1c9e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:47.047091 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb95df78-0934-43c9-bd04-d0e1f25e1c9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:47.048848 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb95df78-0934-43c9-bd04-d0e1f25e1c9e None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:47.073237 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 128/249] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:11:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:11:47.081190 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb95df78-0934-43c9-bd04-d0e1f25e1c9e None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:47.148334 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9a87f4-9182-4f4e-9920-a7b2cb6292a7 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:47.273441 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:47.274495 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae850f7f-1019-4d2c-b9c9-f56a8f98174e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:47.275205 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae850f7f-1019-4d2c-b9c9-f56a8f98174e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:47.275538 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae850f7f-1019-4d2c-b9c9-f56a8f98174e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:47.275815 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae850f7f-1019-4d2c-b9c9-f56a8f98174e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:47.276693 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae850f7f-1019-4d2c-b9c9-f56a8f98174e None admin] RBAC: Adding query filter params (role.id=5efb4931668c4edf87773a1f5d47f005, scope.project.id=ee325f0b1b08455686f9e0a696ca57c8, user.id=ca6ab7a1cd61437288936f3c5c0f2494) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:47.277006 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae850f7f-1019-4d2c-b9c9-f56a8f98174e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:47.278578 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae850f7f-1019-4d2c-b9c9-f56a8f98174e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:47.287837 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 122/250] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:11:46 2022] GET /identity/v3/role_assignments?user.id=ca6ab7a1cd61437288936f3c5c0f2494&scope.project.id=ee325f0b1b08455686f9e0a696ca57c8&role.id=5efb4931668c4edf87773a1f5d47f005 => generated 259 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:47.400640 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb95df78-0934-43c9-bd04-d0e1f25e1c9e None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:47.484701 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcace10-a69a-4223-8782-9a22f73881c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:47.486335 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcace10-a69a-4223-8782-9a22f73881c9 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:47.486785 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcace10-a69a-4223-8782-9a22f73881c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:47.487291 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcace10-a69a-4223-8782-9a22f73881c9 None admin] PATH_INFO: `/v3/users/cd79af98206947fdb9a546a7cb70e1a6` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:47.489698 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcace10-a69a-4223-8782-9a22f73881c9 None admin] RBAC: Authorizing `identity:get_user(user_id=cd79af98206947fdb9a546a7cb70e1a6)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:47.492629 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcace10-a69a-4223-8782-9a22f73881c9 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:47.496274 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 129/251] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:11:47 2022] GET /identity/v3/users/cd79af98206947fdb9a546a7cb70e1a6 => generated 283 bytes in 416 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:47.505285 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae850f7f-1019-4d2c-b9c9-f56a8f98174e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:47.526858 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53144b26-dbb4-4465-88ee-991c5a8be3fd None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:47.527337 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53144b26-dbb4-4465-88ee-991c5a8be3fd None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:47.527810 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53144b26-dbb4-4465-88ee-991c5a8be3fd None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:47.530579 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 130/252] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:11:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:11:47.537807 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca068f3-b48b-4f8f-9693-84fedcf67549 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:47.538310 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca068f3-b48b-4f8f-9693-84fedcf67549 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:47.538806 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca068f3-b48b-4f8f-9693-84fedcf67549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:47.579629 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-cca068f3-b48b-4f8f-9693-84fedcf67549 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:11:47.580951 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-cca068f3-b48b-4f8f-9693-84fedcf67549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:11:47.691934 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 131/253] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:11:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:11:47.728929 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca068f3-b48b-4f8f-9693-84fedcf67549 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:47.776501 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae850f7f-1019-4d2c-b9c9-f56a8f98174e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:47.782081 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:47.880067 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca068f3-b48b-4f8f-9693-84fedcf67549 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:47.898146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:47.915677 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ab3c86-ad7e-4858-b5dc-d8dd874e4457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:47.916659 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab3c86-ad7e-4858-b5dc-d8dd874e4457 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:47.917101 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab3c86-ad7e-4858-b5dc-d8dd874e4457 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:47.917528 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab3c86-ad7e-4858-b5dc-d8dd874e4457 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:47.934249 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43788a0b-3874-4c0b-85d9-0fbf035ea0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:47.935179 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43788a0b-3874-4c0b-85d9-0fbf035ea0fd None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:47.935476 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43788a0b-3874-4c0b-85d9-0fbf035ea0fd None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:47.935893 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43788a0b-3874-4c0b-85d9-0fbf035ea0fd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:47.962489 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43788a0b-3874-4c0b-85d9-0fbf035ea0fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:11:47.978358 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-43788a0b-3874-4c0b-85d9-0fbf035ea0fd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:11:47.978358 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:11:47.978358 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:11:47.978358 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:11:47.978358 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:11:47.978358 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:11:47.978358 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:11:47.978358 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:11:47.978358 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:11:47.978358 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:11:47.978358 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:11:47.978358 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:11:47.978358 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:11:47.978358 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:11:47.978358 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:11:47.978358 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:11:47.979346 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:11:47.979346 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:11:47.979346 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:11:47.979346 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:11:47.979346 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:11:47.979346 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:11:47.979346 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:11:47.979346 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:11:47.979346 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:11:47.979346 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:11:47.979346 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:11:47.979346 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:11:47.979346 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:11:47.979346 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:11:47.979346 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:11:47.979346 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:11:47.980198 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:11:47.980198 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:11:47.980198 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:11:47.980198 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:11:47.980198 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:11:47.980198 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:11:47.980198 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:11:47.980198 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:11:47.980198 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:11:47.980198 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:11:47.980198 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:11:47.980198 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:11:47.985578 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 132/254] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:11:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:11:47.988997 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eabcf7-9fc1-4151-9c93-55499527dc63 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:47.989325 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eabcf7-9fc1-4151-9c93-55499527dc63 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:47.989614 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eabcf7-9fc1-4151-9c93-55499527dc63 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:47.991170 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 133/255] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:11:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:11:47.991971 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eabcf7-9fc1-4151-9c93-55499527dc63 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:48.129525 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ab3c86-ad7e-4858-b5dc-d8dd874e4457 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:48.132522 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ab3c86-ad7e-4858-b5dc-d8dd874e4457 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:48.348679 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eabcf7-9fc1-4151-9c93-55499527dc63 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:48.418719 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:48.474742 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c23c6a-474d-4808-96d3-7a4bd8de6dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:48.475891 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c23c6a-474d-4808-96d3-7a4bd8de6dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:48.476388 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c23c6a-474d-4808-96d3-7a4bd8de6dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:48.476873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c23c6a-474d-4808-96d3-7a4bd8de6dd3 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:48.478395 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c23c6a-474d-4808-96d3-7a4bd8de6dd3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:48.479892 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c23c6a-474d-4808-96d3-7a4bd8de6dd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:48.482825 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c23c6a-474d-4808-96d3-7a4bd8de6dd3 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:48.489179 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 123/256] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:11:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 984 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:11:48.495270 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79202aaf-d64f-492e-92e7-3a1d7ed7351a None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:48.495614 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79202aaf-d64f-492e-92e7-3a1d7ed7351a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:48.495851 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79202aaf-d64f-492e-92e7-3a1d7ed7351a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:48.501256 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 134/257] 10.222.0.87 () {56 vars in 1101 bytes} [Tue Jun 21 18:11:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 509 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:48.502365 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c23c6a-474d-4808-96d3-7a4bd8de6dd3 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:48.510063 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:48.538819 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-79202aaf-d64f-492e-92e7-3a1d7ed7351a None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:11:48.540067 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-79202aaf-d64f-492e-92e7-3a1d7ed7351a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:11:48.733699 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c23c6a-474d-4808-96d3-7a4bd8de6dd3 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:48.767106 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 124/258] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:11:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:11:48.768358 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79202aaf-d64f-492e-92e7-3a1d7ed7351a None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:48.826175 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb37126-1197-4781-af94-d5cca5edda5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:48.827583 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb37126-1197-4781-af94-d5cca5edda5d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:48.828264 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb37126-1197-4781-af94-d5cca5edda5d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:48.828715 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb37126-1197-4781-af94-d5cca5edda5d None admin] PATH_INFO: `/v3/projects/ee325f0b1b08455686f9e0a696ca57c8` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:48.841848 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb37126-1197-4781-af94-d5cca5edda5d None admin] RBAC: Authorizing `identity:get_project(project_id=ee325f0b1b08455686f9e0a696ca57c8)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:48.842570 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb37126-1197-4781-af94-d5cca5edda5d None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:48.846058 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 135/259] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:11:48 2022] GET /identity/v3/projects/ee325f0b1b08455686f9e0a696ca57c8 => generated 302 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:48.853209 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb37126-1197-4781-af94-d5cca5edda5d None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:48.968105 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79202aaf-d64f-492e-92e7-3a1d7ed7351a None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:48.988297 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:49.008218 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:49.016045 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea74817d-95aa-4b2f-a93e-0e548454d07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:49.017262 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea74817d-95aa-4b2f-a93e-0e548454d07e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:49.017546 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea74817d-95aa-4b2f-a93e-0e548454d07e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:49.017861 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea74817d-95aa-4b2f-a93e-0e548454d07e None admin] PATH_INFO: `/v3/users/99487f792aae445d847c795ec9018376` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:49.019259 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea74817d-95aa-4b2f-a93e-0e548454d07e None admin] RBAC: Authorizing `identity:get_user(user_id=99487f792aae445d847c795ec9018376)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:49.020876 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea74817d-95aa-4b2f-a93e-0e548454d07e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:49.022885 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 125/260] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:11:48 2022] GET /identity/v3/users/99487f792aae445d847c795ec9018376 => generated 297 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:49.024038 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaffc873-e4d3-426f-95db-4c483855a802 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:49.024325 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaffc873-e4d3-426f-95db-4c483855a802 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:49.024788 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaffc873-e4d3-426f-95db-4c483855a802 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:49.026732 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 126/261] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:11:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:11:49.027409 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaffc873-e4d3-426f-95db-4c483855a802 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:49.095045 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb37126-1197-4781-af94-d5cca5edda5d None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:49.156937 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545d2b6-64d8-464c-964a-f219e678a6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:49.157881 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545d2b6-64d8-464c-964a-f219e678a6a0 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:49.158286 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545d2b6-64d8-464c-964a-f219e678a6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:49.158635 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545d2b6-64d8-464c-964a-f219e678a6a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:49.160186 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d545d2b6-64d8-464c-964a-f219e678a6a0 None admin] RBAC: Adding query filter params (role.id=98f0943a610f4a9e9f8de54f6e9f717d, scope.project.id=ee325f0b1b08455686f9e0a696ca57c8, user.id=cd79af98206947fdb9a546a7cb70e1a6) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:49.160629 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d545d2b6-64d8-464c-964a-f219e678a6a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:49.162659 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d545d2b6-64d8-464c-964a-f219e678a6a0 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:49.171312 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 136/262] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:11:48 2022] GET /identity/v3/role_assignments?user.id=cd79af98206947fdb9a546a7cb70e1a6&scope.project.id=ee325f0b1b08455686f9e0a696ca57c8&role.id=98f0943a610f4a9e9f8de54f6e9f717d => generated 608 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:49.172403 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545d2b6-64d8-464c-964a-f219e678a6a0 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:49.243328 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaffc873-e4d3-426f-95db-4c483855a802 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:49.366522 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e51a945-9a1d-4881-8450-df53ae9d6f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:49.367403 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51a945-9a1d-4881-8450-df53ae9d6f98 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:49.367700 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51a945-9a1d-4881-8450-df53ae9d6f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:49.367987 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51a945-9a1d-4881-8450-df53ae9d6f98 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:49.382538 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e51a945-9a1d-4881-8450-df53ae9d6f98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:11:49.391612 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-1e51a945-9a1d-4881-8450-df53ae9d6f98 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:11:49.391612 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:11:49.391612 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:11:49.391612 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:11:49.391612 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:11:49.391612 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:11:49.391612 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:11:49.391612 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:11:49.391612 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:11:49.391612 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:11:49.391612 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:11:49.391612 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:11:49.391612 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:11:49.391612 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:11:49.391612 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:11:49.391612 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:11:49.392576 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:11:49.392576 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:11:49.392576 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:11:49.392576 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:11:49.392576 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:11:49.392576 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:11:49.392576 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:11:49.392576 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:11:49.392576 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:11:49.392576 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:11:49.392576 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:11:49.392576 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:11:49.392576 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:11:49.392576 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:11:49.392576 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:11:49.392576 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:11:49.393683 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:11:49.393683 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:11:49.393683 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:11:49.393683 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:11:49.393683 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:11:49.393683 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:11:49.393683 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:11:49.393683 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:11:49.393683 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:11:49.393683 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:11:49.393683 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:11:49.393683 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:11:49.396462 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545d2b6-64d8-464c-964a-f219e678a6a0 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:49.397147 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 127/263] 10.222.0.87 () {56 vars in 1114 bytes} [Tue Jun 21 18:11:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:11:49.398796 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128570e-205d-4729-92d0-ff70f4cc47ed None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:49.399368 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128570e-205d-4729-92d0-ff70f4cc47ed None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:49.399848 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128570e-205d-4729-92d0-ff70f4cc47ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:49.420253 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-9128570e-205d-4729-92d0-ff70f4cc47ed None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:11:49.421880 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9128570e-205d-4729-92d0-ff70f4cc47ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:11:49.435305 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:49.480558 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79678173-54be-472e-a7f7-096cda3b8dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:49.481673 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79678173-54be-472e-a7f7-096cda3b8dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:49.481907 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79678173-54be-472e-a7f7-096cda3b8dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:49.482400 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79678173-54be-472e-a7f7-096cda3b8dd1 None admin] PATH_INFO: `/v3/projects/ee325f0b1b08455686f9e0a696ca57c8` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:49.485395 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79678173-54be-472e-a7f7-096cda3b8dd1 None admin] RBAC: Authorizing `identity:get_project(project_id=ee325f0b1b08455686f9e0a696ca57c8)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:49.489129 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79678173-54be-472e-a7f7-096cda3b8dd1 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:49.492552 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 137/264] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:11:49 2022] GET /identity/v3/projects/ee325f0b1b08455686f9e0a696ca57c8 => generated 302 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:49.498235 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4520e3-562f-4af6-b009-67de5f3438cb None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:49.498633 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4520e3-562f-4af6-b009-67de5f3438cb None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:49.499206 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4520e3-562f-4af6-b009-67de5f3438cb None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:49.502602 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 138/265] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:11:49 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:11:49.503822 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4520e3-562f-4af6-b009-67de5f3438cb None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:49.540036 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:49.558339 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 128/266] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:11:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:11:49.559377 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9128570e-205d-4729-92d0-ff70f4cc47ed None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:49.715709 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4520e3-562f-4af6-b009-67de5f3438cb None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:49.755291 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9128570e-205d-4729-92d0-ff70f4cc47ed None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:49.784872 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0c88cb-96ee-417c-a381-6d6df79b9863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:49.786270 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c88cb-96ee-417c-a381-6d6df79b9863 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:49.786690 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c88cb-96ee-417c-a381-6d6df79b9863 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:49.787155 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c88cb-96ee-417c-a381-6d6df79b9863 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:49.789385 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0c88cb-96ee-417c-a381-6d6df79b9863 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:49.789385 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0c88cb-96ee-417c-a381-6d6df79b9863 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:49.790888 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0c88cb-96ee-417c-a381-6d6df79b9863 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:49.808127 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 139/267] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:11:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:49.813214 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15127523-b36c-4b7d-a654-78eba7dde6be None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:49.813214 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15127523-b36c-4b7d-a654-78eba7dde6be None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:49.813214 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15127523-b36c-4b7d-a654-78eba7dde6be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:49.833863 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954aefe2-0b3b-4031-91d3-de310d3f8c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:49.835196 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954aefe2-0b3b-4031-91d3-de310d3f8c20 None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:49.835658 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954aefe2-0b3b-4031-91d3-de310d3f8c20 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:49.836113 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954aefe2-0b3b-4031-91d3-de310d3f8c20 None admin] PATH_INFO: `/v3/projects/ee325f0b1b08455686f9e0a696ca57c8/users/99487f792aae445d847c795ec9018376/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:49.839529 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954aefe2-0b3b-4031-91d3-de310d3f8c20 None admin] RBAC: Authorizing `identity:create_grant(project_id=ee325f0b1b08455686f9e0a696ca57c8, user_id=99487f792aae445d847c795ec9018376, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:49.841961 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954aefe2-0b3b-4031-91d3-de310d3f8c20 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:49.844523 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-15127523-b36c-4b7d-a654-78eba7dde6be None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:11:49.848113 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-15127523-b36c-4b7d-a654-78eba7dde6be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:11:49.865918 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 129/268] 10.222.0.87 () {58 vars in 1439 bytes} [Tue Jun 21 18:11:49 2022] PUT /identity/v3/projects/ee325f0b1b08455686f9e0a696ca57c8/users/99487f792aae445d847c795ec9018376/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:11:49.867109 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954aefe2-0b3b-4031-91d3-de310d3f8c20 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:49.939000 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:49.996378 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:50.009766 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 140/269] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:11:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:11:50.018365 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15127523-b36c-4b7d-a654-78eba7dde6be None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:50.130996 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954aefe2-0b3b-4031-91d3-de310d3f8c20 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:50.222594 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4205abd0-045d-4856-99c5-a452f3a62b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:50.224030 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205abd0-045d-4856-99c5-a452f3a62b40 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:50.224549 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205abd0-045d-4856-99c5-a452f3a62b40 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:50.225052 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205abd0-045d-4856-99c5-a452f3a62b40 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:50.252736 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4205abd0-045d-4856-99c5-a452f3a62b40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:11:50.265285 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-4205abd0-045d-4856-99c5-a452f3a62b40 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:11:50.265285 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:11:50.265285 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:11:50.265285 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:11:50.265285 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:11:50.265285 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:11:50.265285 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:11:50.265285 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:11:50.265285 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:11:50.265285 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:11:50.265285 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:11:50.265285 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:11:50.265285 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:11:50.265285 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:11:50.265285 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:11:50.265285 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:11:50.267325 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:11:50.267325 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:11:50.267325 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:11:50.267325 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:11:50.267325 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:11:50.267325 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:11:50.267325 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:11:50.267325 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:11:50.267325 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:11:50.267325 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:11:50.267325 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:11:50.267325 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:11:50.267325 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:11:50.267325 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:11:50.267325 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:11:50.267325 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:11:50.269792 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:11:50.269792 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:11:50.269792 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:11:50.269792 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:11:50.269792 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:11:50.269792 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:11:50.269792 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:11:50.269792 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:11:50.269792 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:11:50.269792 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:11:50.269792 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:11:50.269792 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:11:50.269792 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 130/270] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:11:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 400 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:11:50.269792 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4205abd0-045d-4856-99c5-a452f3a62b40 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:50.309113 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15127523-b36c-4b7d-a654-78eba7dde6be None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:50.387024 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f39742-7f78-4184-a6b7-df3675094fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:50.388429 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f39742-7f78-4184-a6b7-df3675094fdd None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:50.388885 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f39742-7f78-4184-a6b7-df3675094fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:50.389325 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f39742-7f78-4184-a6b7-df3675094fdd None admin] PATH_INFO: `/v3/users/cd79af98206947fdb9a546a7cb70e1a6` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:50.391765 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f39742-7f78-4184-a6b7-df3675094fdd None admin] RBAC: Authorizing `identity:get_user(user_id=cd79af98206947fdb9a546a7cb70e1a6)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:50.396428 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f39742-7f78-4184-a6b7-df3675094fdd None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:50.398819 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 141/271] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:11:50 2022] GET /identity/v3/users/cd79af98206947fdb9a546a7cb70e1a6 => generated 283 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:50.400006 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f465da5-8cb3-4336-b2a1-3dd84d745c3f None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:50.400393 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f465da5-8cb3-4336-b2a1-3dd84d745c3f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:50.400863 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f465da5-8cb3-4336-b2a1-3dd84d745c3f None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:50.403418 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 142/272] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:11:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:11:50.404691 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f465da5-8cb3-4336-b2a1-3dd84d745c3f None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:50.476724 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4205abd0-045d-4856-99c5-a452f3a62b40 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:50.491960 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:50.518389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:50.526655 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fff1747-d215-4004-9dc6-505e5e810821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:50.527557 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff1747-d215-4004-9dc6-505e5e810821 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:50.528899 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff1747-d215-4004-9dc6-505e5e810821 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:50.529227 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff1747-d215-4004-9dc6-505e5e810821 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:50.531198 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fff1747-d215-4004-9dc6-505e5e810821 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:50.532798 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fff1747-d215-4004-9dc6-505e5e810821 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:50.544201 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 131/273] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:11:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:50.544201 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fff1747-d215-4004-9dc6-505e5e810821 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:50.642371 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f465da5-8cb3-4336-b2a1-3dd84d745c3f None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:50.730976 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccc7ec5-1cd8-4233-8aaa-44bb369db3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:50.731761 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc7ec5-1cd8-4233-8aaa-44bb369db3c9 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:50.732758 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc7ec5-1cd8-4233-8aaa-44bb369db3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:50.733619 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc7ec5-1cd8-4233-8aaa-44bb369db3c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:50.734523 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccc7ec5-1cd8-4233-8aaa-44bb369db3c9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:50.734831 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccc7ec5-1cd8-4233-8aaa-44bb369db3c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:50.735066 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fff1747-d215-4004-9dc6-505e5e810821 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:50.736539 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccc7ec5-1cd8-4233-8aaa-44bb369db3c9 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:50.755530 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 143/274] 10.222.0.87 () {56 vars in 1101 bytes} [Tue Jun 21 18:11:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:50.758995 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb90340-f8ee-4d7d-9024-98e1c1c80f59 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:50.759929 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb90340-f8ee-4d7d-9024-98e1c1c80f59 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:50.760818 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb90340-f8ee-4d7d-9024-98e1c1c80f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:50.793461 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-5fb90340-f8ee-4d7d-9024-98e1c1c80f59 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:11:50.797211 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5fb90340-f8ee-4d7d-9024-98e1c1c80f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:11:50.823350 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba79de27-e2e3-4b2a-aa19-8cbc499d57fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:50.823350 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba79de27-e2e3-4b2a-aa19-8cbc499d57fc None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:50.823612 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba79de27-e2e3-4b2a-aa19-8cbc499d57fc None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:50.824045 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba79de27-e2e3-4b2a-aa19-8cbc499d57fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:50.934292 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:50.956624 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 144/275] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:11:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:11:50.963810 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb90340-f8ee-4d7d-9024-98e1c1c80f59 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:50.986675 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba79de27-e2e3-4b2a-aa19-8cbc499d57fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:50.990503 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba79de27-e2e3-4b2a-aa19-8cbc499d57fc None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:51.042260 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:51.181583 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb90340-f8ee-4d7d-9024-98e1c1c80f59 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:51.231540 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ddbedb-0a2a-40d1-8ea5-9798eeebc648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:51.232730 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ddbedb-0a2a-40d1-8ea5-9798eeebc648 None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:51.233530 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ddbedb-0a2a-40d1-8ea5-9798eeebc648 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:51.234093 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ddbedb-0a2a-40d1-8ea5-9798eeebc648 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:51.235459 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ddbedb-0a2a-40d1-8ea5-9798eeebc648 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:51.238250 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ddbedb-0a2a-40d1-8ea5-9798eeebc648 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:51.244186 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 132/276] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:11:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 708 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:11:51.246754 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba79de27-e2e3-4b2a-aa19-8cbc499d57fc None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:51.278936 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 145/277] 10.222.0.87 () {60 vars in 1132 bytes} [Tue Jun 21 18:11:50 2022] POST /identity/v3/users => generated 293 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:11:51.280365 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ddbedb-0a2a-40d1-8ea5-9798eeebc648 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:51.431180 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:51.470489 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:51.484824 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba79de27-e2e3-4b2a-aa19-8cbc499d57fc None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:51.499907 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ddbedb-0a2a-40d1-8ea5-9798eeebc648 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:51.566263 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc75fc-0cc7-4980-92a1-6e1786154e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:51.566739 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc75fc-0cc7-4980-92a1-6e1786154e0a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:51.567151 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc75fc-0cc7-4980-92a1-6e1786154e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:51.567655 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc75fc-0cc7-4980-92a1-6e1786154e0a None admin] PATH_INFO: `/v3/users/0de5ac62220f43ee93507be46407afed` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:51.579119 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95a5302-c6eb-4d1d-8fd2-9e96425f4122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:51.580201 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a5302-c6eb-4d1d-8fd2-9e96425f4122 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:51.580720 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a5302-c6eb-4d1d-8fd2-9e96425f4122 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:51.581158 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a5302-c6eb-4d1d-8fd2-9e96425f4122 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:51.600451 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95a5302-c6eb-4d1d-8fd2-9e96425f4122 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:11:51.600696 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cc75fc-0cc7-4980-92a1-6e1786154e0a None admin] RBAC: Authorizing `identity:get_user(user_id=0de5ac62220f43ee93507be46407afed)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:51.603491 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cc75fc-0cc7-4980-92a1-6e1786154e0a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:51.607119 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 133/278] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:11:51 2022] GET /identity/v3/users/0de5ac62220f43ee93507be46407afed => generated 293 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:51.609045 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc75fc-0cc7-4980-92a1-6e1786154e0a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:51.621862 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-f95a5302-c6eb-4d1d-8fd2-9e96425f4122 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:11:51.621862 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:11:51.621862 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:11:51.621862 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:11:51.621862 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:11:51.621862 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:11:51.621862 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:11:51.621862 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:11:51.621862 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:11:51.621862 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:11:51.621862 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:11:51.621862 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:11:51.621862 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:11:51.621862 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:11:51.621862 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:11:51.621862 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:11:51.624058 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:11:51.624058 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:11:51.624058 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:11:51.624058 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:11:51.624058 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:11:51.624058 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:11:51.624058 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:11:51.624058 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:11:51.624058 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:11:51.624058 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:11:51.624058 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:11:51.624058 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:11:51.624058 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:11:51.624058 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:11:51.624058 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:11:51.624058 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:11:51.626943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:11:51.626943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:11:51.626943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:11:51.626943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:11:51.626943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:11:51.626943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:11:51.626943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:11:51.626943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:11:51.626943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:11:51.626943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:11:51.626943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:11:51.626943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:11:51.626943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 146/279] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:11:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:11:51.626943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95a5302-c6eb-4d1d-8fd2-9e96425f4122 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:51.853356 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc75fc-0cc7-4980-92a1-6e1786154e0a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:51.879575 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95a5302-c6eb-4d1d-8fd2-9e96425f4122 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:51.930790 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea021537-44ab-48e1-8ae0-e7701c610c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:51.932446 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea021537-44ab-48e1-8ae0-e7701c610c55 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:51.933356 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea021537-44ab-48e1-8ae0-e7701c610c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:51.934121 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea021537-44ab-48e1-8ae0-e7701c610c55 None admin] PATH_INFO: `/v3/projects/ee325f0b1b08455686f9e0a696ca57c8` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:51.936622 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea021537-44ab-48e1-8ae0-e7701c610c55 None admin] RBAC: Authorizing `identity:get_project(project_id=ee325f0b1b08455686f9e0a696ca57c8)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:51.940562 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea021537-44ab-48e1-8ae0-e7701c610c55 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:51.944147 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 134/280] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:11:51 2022] GET /identity/v3/projects/ee325f0b1b08455686f9e0a696ca57c8 => generated 302 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:51.946035 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea021537-44ab-48e1-8ae0-e7701c610c55 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:51.951765 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:51.963069 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddb40c6-1e73-464d-8037-697ab89c119d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:51.966562 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb40c6-1e73-464d-8037-697ab89c119d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:51.967451 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb40c6-1e73-464d-8037-697ab89c119d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:51.968150 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb40c6-1e73-464d-8037-697ab89c119d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:51.969529 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddb40c6-1e73-464d-8037-697ab89c119d None admin] RBAC: Adding query filter params (role.id=da6d4a0ab8c04809ba120170fb4ae8a9, user.id=0de5ac62220f43ee93507be46407afed, scope.system=all) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:51.970229 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddb40c6-1e73-464d-8037-697ab89c119d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:51.973400 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddb40c6-1e73-464d-8037-697ab89c119d None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:51.990801 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 147/281] 10.222.0.87 () {56 vars in 1308 bytes} [Tue Jun 21 18:11:51 2022] GET /identity/v3/role_assignments?user.id=0de5ac62220f43ee93507be46407afed&scope.system=all&role.id=da6d4a0ab8c04809ba120170fb4ae8a9 => generated 226 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:51.992103 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddb40c6-1e73-464d-8037-697ab89c119d None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:52.026244 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:52.163027 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddb40c6-1e73-464d-8037-697ab89c119d None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:52.219933 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aef1de-146c-490f-a5d4-727c49022b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:52.221295 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aef1de-146c-490f-a5d4-727c49022b14 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:52.221898 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aef1de-146c-490f-a5d4-727c49022b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:52.222451 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aef1de-146c-490f-a5d4-727c49022b14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:52.223561 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aef1de-146c-490f-a5d4-727c49022b14 None admin] RBAC: Adding query filter params (role.id=46e77bb17fc3465496a7a3a29d210042, scope.project.id=ee325f0b1b08455686f9e0a696ca57c8, user.id=cd79af98206947fdb9a546a7cb70e1a6) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:52.224048 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aef1de-146c-490f-a5d4-727c49022b14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:52.225951 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aef1de-146c-490f-a5d4-727c49022b14 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:52.235889 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 148/282] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:11:51 2022] GET /identity/v3/role_assignments?user.id=cd79af98206947fdb9a546a7cb70e1a6&scope.project.id=ee325f0b1b08455686f9e0a696ca57c8&role.id=46e77bb17fc3465496a7a3a29d210042 => generated 608 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:52.289765 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea021537-44ab-48e1-8ae0-e7701c610c55 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:52.392705 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c4cb5a-cc04-4ca7-8020-3123aad1b541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:52.394771 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4cb5a-cc04-4ca7-8020-3123aad1b541 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:52.395601 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4cb5a-cc04-4ca7-8020-3123aad1b541 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:52.396401 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4cb5a-cc04-4ca7-8020-3123aad1b541 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:52.398111 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c4cb5a-cc04-4ca7-8020-3123aad1b541 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:52.398698 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c4cb5a-cc04-4ca7-8020-3123aad1b541 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:52.400940 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c4cb5a-cc04-4ca7-8020-3123aad1b541 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:52.414474 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 135/283] 10.222.0.87 () {56 vars in 1101 bytes} [Tue Jun 21 18:11:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 469 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:52.423194 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aef1de-146c-490f-a5d4-727c49022b14 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:52.575387 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:52.632690 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aef1de-146c-490f-a5d4-727c49022b14 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:52.694582 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4719a23-1a73-4cb1-a04a-86c44bcaca01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:52.695650 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4719a23-1a73-4cb1-a04a-86c44bcaca01 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:52.695939 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4719a23-1a73-4cb1-a04a-86c44bcaca01 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:52.696356 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4719a23-1a73-4cb1-a04a-86c44bcaca01 None admin] PATH_INFO: `/v3/users/ca6ab7a1cd61437288936f3c5c0f2494` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:52.697909 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4719a23-1a73-4cb1-a04a-86c44bcaca01 None admin] RBAC: Authorizing `identity:get_user(user_id=ca6ab7a1cd61437288936f3c5c0f2494)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:52.699408 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4719a23-1a73-4cb1-a04a-86c44bcaca01 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:52.701788 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 149/284] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:11:52 2022] GET /identity/v3/users/ca6ab7a1cd61437288936f3c5c0f2494 => generated 297 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:11:52.707760 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c4cb5a-cc04-4ca7-8020-3123aad1b541 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:52.865576 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:52.917579 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c4cb5a-cc04-4ca7-8020-3123aad1b541 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:52.964986 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a27a7-0a82-460c-9a3e-cee410869793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:52.966440 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a27a7-0a82-460c-9a3e-cee410869793 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:52.966953 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a27a7-0a82-460c-9a3e-cee410869793 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:52.967429 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a27a7-0a82-460c-9a3e-cee410869793 None admin] PATH_INFO: `/v3/projects/ee325f0b1b08455686f9e0a696ca57c8` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:52.969269 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19a27a7-0a82-460c-9a3e-cee410869793 None admin] RBAC: Authorizing `identity:get_project(project_id=ee325f0b1b08455686f9e0a696ca57c8)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:52.971439 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19a27a7-0a82-460c-9a3e-cee410869793 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:52.974260 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 136/285] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:11:52 2022] GET /identity/v3/projects/ee325f0b1b08455686f9e0a696ca57c8 => generated 302 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:52.981724 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4719a23-1a73-4cb1-a04a-86c44bcaca01 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:53.145902 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4719a23-1a73-4cb1-a04a-86c44bcaca01 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:53.193349 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b66123-70a0-413b-9907-a9bde4fec8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:53.194306 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b66123-70a0-413b-9907-a9bde4fec8cb None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:53.194594 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b66123-70a0-413b-9907-a9bde4fec8cb None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:53.194874 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b66123-70a0-413b-9907-a9bde4fec8cb None admin] PATH_INFO: `/v3/projects/ee325f0b1b08455686f9e0a696ca57c8/users/ca6ab7a1cd61437288936f3c5c0f2494/roles/5efb4931668c4edf87773a1f5d47f005` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:53.202762 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b66123-70a0-413b-9907-a9bde4fec8cb None admin] RBAC: Authorizing `identity:create_grant(project_id=ee325f0b1b08455686f9e0a696ca57c8, user_id=ca6ab7a1cd61437288936f3c5c0f2494, role_id=5efb4931668c4edf87773a1f5d47f005)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:53.204421 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b66123-70a0-413b-9907-a9bde4fec8cb None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:53.224459 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 150/286] 10.222.0.87 () {58 vars in 1439 bytes} [Tue Jun 21 18:11:52 2022] PUT /identity/v3/projects/ee325f0b1b08455686f9e0a696ca57c8/users/ca6ab7a1cd61437288936f3c5c0f2494/roles/5efb4931668c4edf87773a1f5d47f005 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:11:53.599191 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0548c-a4b2-4eba-bc8d-94c4dc677201 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:53.600722 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0548c-a4b2-4eba-bc8d-94c4dc677201 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:53.601385 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0548c-a4b2-4eba-bc8d-94c4dc677201 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:53.604497 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 137/287] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:11:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:11:53.614985 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c47423-b649-493a-aebd-ad4398081435 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:53.615471 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c47423-b649-493a-aebd-ad4398081435 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:53.615950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c47423-b649-493a-aebd-ad4398081435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:53.640904 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:53.652005 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-99c47423-b649-493a-aebd-ad4398081435 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:11:53.654160 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-99c47423-b649-493a-aebd-ad4398081435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:11:53.801834 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 151/288] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:11:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:11:53.836706 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad0548c-a4b2-4eba-bc8d-94c4dc677201 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:54.074932 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad0548c-a4b2-4eba-bc8d-94c4dc677201 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:54.144604 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62ab3b0-9df5-46ea-b02a-4ef3aec89bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:54.146459 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62ab3b0-9df5-46ea-b02a-4ef3aec89bb9 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:54.147284 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62ab3b0-9df5-46ea-b02a-4ef3aec89bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:54.148056 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62ab3b0-9df5-46ea-b02a-4ef3aec89bb9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:54.152132 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:54.183522 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62ab3b0-9df5-46ea-b02a-4ef3aec89bb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:11:54.203804 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-f62ab3b0-9df5-46ea-b02a-4ef3aec89bb9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:11:54.203804 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:11:54.203804 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:11:54.203804 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:11:54.203804 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:11:54.203804 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:11:54.203804 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:11:54.203804 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:11:54.203804 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:11:54.203804 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:11:54.203804 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:11:54.203804 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:11:54.203804 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:11:54.203804 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:11:54.203804 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:11:54.203804 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:11:54.205689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:11:54.205689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:11:54.205689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:11:54.205689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:11:54.205689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:11:54.205689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:11:54.205689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:11:54.205689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:11:54.205689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:11:54.205689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:11:54.205689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:11:54.205689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:11:54.205689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:11:54.205689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:11:54.205689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:11:54.205689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:11:54.207232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:11:54.207232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:11:54.207232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:11:54.207232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:11:54.207232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:11:54.207232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:11:54.207232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:11:54.207232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:11:54.207232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:11:54.207232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:11:54.207232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:11:54.207232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:11:54.207232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 138/289] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:11:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:11:54.216999 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c47423-b649-493a-aebd-ad4398081435 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:54.372903 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c47423-b649-493a-aebd-ad4398081435 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:54.414085 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f4d450-9464-4fd1-a450-b756e79a1d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:54.414832 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4d450-9464-4fd1-a450-b756e79a1d79 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:54.415119 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4d450-9464-4fd1-a450-b756e79a1d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:54.415447 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4d450-9464-4fd1-a450-b756e79a1d79 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:54.416236 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f4d450-9464-4fd1-a450-b756e79a1d79 None admin] RBAC: Adding query filter params (name=member) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:54.416546 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f4d450-9464-4fd1-a450-b756e79a1d79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:54.418129 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f4d450-9464-4fd1-a450-b756e79a1d79 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:54.430063 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:54.432545 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 152/290] 10.222.0.87 () {56 vars in 1101 bytes} [Tue Jun 21 18:11:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:54.434332 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62ab3b0-9df5-46ea-b02a-4ef3aec89bb9 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:54.590542 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62ab3b0-9df5-46ea-b02a-4ef3aec89bb9 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:54.643756 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e87e583-bceb-4fe0-927c-0dd93be832ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:54.644635 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e87e583-bceb-4fe0-927c-0dd93be832ce None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:54.644923 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e87e583-bceb-4fe0-927c-0dd93be832ce None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:54.645204 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e87e583-bceb-4fe0-927c-0dd93be832ce None admin] PATH_INFO: `/v3/users/99487f792aae445d847c795ec9018376` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:54.646613 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e87e583-bceb-4fe0-927c-0dd93be832ce None admin] RBAC: Authorizing `identity:get_user(user_id=99487f792aae445d847c795ec9018376)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:54.648154 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e87e583-bceb-4fe0-927c-0dd93be832ce None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:54.650155 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 139/291] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:11:54 2022] GET /identity/v3/users/99487f792aae445d847c795ec9018376 => generated 297 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:54.656757 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f4d450-9464-4fd1-a450-b756e79a1d79 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:54.922600 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f4d450-9464-4fd1-a450-b756e79a1d79 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:54.976502 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8333983-7953-4e08-93a4-0db9109ee519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:54.977203 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8333983-7953-4e08-93a4-0db9109ee519 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:54.977534 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8333983-7953-4e08-93a4-0db9109ee519 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:54.977838 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8333983-7953-4e08-93a4-0db9109ee519 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:55.081142 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:55.090221 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8333983-7953-4e08-93a4-0db9109ee519 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:55.092178 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8333983-7953-4e08-93a4-0db9109ee519 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:55.121798 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf2ed90-1f0f-42a6-b892-bbbc78a9a776 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:55.122211 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf2ed90-1f0f-42a6-b892-bbbc78a9a776 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:55.122356 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf2ed90-1f0f-42a6-b892-bbbc78a9a776 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:55.124238 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 140/292] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:11:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:11:55.133688 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f4da3-6d9b-4fb7-8f6e-4e0967304779 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:55.134115 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f4da3-6d9b-4fb7-8f6e-4e0967304779 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:55.134361 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f4da3-6d9b-4fb7-8f6e-4e0967304779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:55.164645 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-956f4da3-6d9b-4fb7-8f6e-4e0967304779 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:11:55.166457 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-956f4da3-6d9b-4fb7-8f6e-4e0967304779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:11:55.310236 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:55.327463 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 153/293] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:11:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 672 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 21 18:11:55.335496 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8333983-7953-4e08-93a4-0db9109ee519 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:55.344029 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 141/294] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:11:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:11:55.345922 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98777ae9-6ef7-4276-9355-7588a5b71baf None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:55.346454 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98777ae9-6ef7-4276-9355-7588a5b71baf None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:55.346967 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98777ae9-6ef7-4276-9355-7588a5b71baf None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:55.349129 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 142/295] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:11:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:11:55.359453 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd026e1f-3529-4aca-af83-1bdd7ee01d00 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:55.360086 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd026e1f-3529-4aca-af83-1bdd7ee01d00 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:55.360585 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd026e1f-3529-4aca-af83-1bdd7ee01d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:55.393605 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-bd026e1f-3529-4aca-af83-1bdd7ee01d00 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:11:55.396065 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-bd026e1f-3529-4aca-af83-1bdd7ee01d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:11:55.507705 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:55.515878 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8333983-7953-4e08-93a4-0db9109ee519 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:55.541707 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 143/296] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:11:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:11:55.541707 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd026e1f-3529-4aca-af83-1bdd7ee01d00 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:55.585898 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d255e01b-9333-4eff-a75d-7e388ceb747f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:55.586841 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255e01b-9333-4eff-a75d-7e388ceb747f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:55.587193 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255e01b-9333-4eff-a75d-7e388ceb747f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:55.587482 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255e01b-9333-4eff-a75d-7e388ceb747f None admin] PATH_INFO: `/v3/projects/ee325f0b1b08455686f9e0a696ca57c8` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:55.589426 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d255e01b-9333-4eff-a75d-7e388ceb747f None admin] RBAC: Authorizing `identity:get_project(project_id=ee325f0b1b08455686f9e0a696ca57c8)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:55.592025 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d255e01b-9333-4eff-a75d-7e388ceb747f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:55.596938 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 154/297] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:11:55 2022] GET /identity/v3/projects/ee325f0b1b08455686f9e0a696ca57c8 => generated 302 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:11:55.597770 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5ab78-c1d3-49c4-9e32-28265b294063 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:55.598264 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5ab78-c1d3-49c4-9e32-28265b294063 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:55.598733 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5ab78-c1d3-49c4-9e32-28265b294063 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:55.601406 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 155/298] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:11:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:11:55.607148 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe5ab78-c1d3-49c4-9e32-28265b294063 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:55.714549 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd026e1f-3529-4aca-af83-1bdd7ee01d00 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:55.757277 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:55.776386 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9209c333-d502-4dcc-928a-ba7b1786c934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:55.777347 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9209c333-d502-4dcc-928a-ba7b1786c934 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:55.777922 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9209c333-d502-4dcc-928a-ba7b1786c934 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:55.778481 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9209c333-d502-4dcc-928a-ba7b1786c934 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:55.802144 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9209c333-d502-4dcc-928a-ba7b1786c934 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:11:55.814686 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-9209c333-d502-4dcc-928a-ba7b1786c934 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:11:55.814686 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:11:55.814686 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:11:55.814686 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:11:55.814686 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:11:55.814686 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:11:55.814686 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:11:55.814686 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:11:55.814686 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:11:55.814686 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:11:55.814686 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:11:55.814686 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:11:55.814686 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:11:55.814686 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:11:55.814686 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:11:55.814686 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:11:55.816402 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:11:55.816402 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:11:55.816402 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:11:55.816402 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:11:55.816402 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:11:55.816402 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:11:55.816402 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:11:55.816402 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:11:55.816402 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:11:55.816402 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:11:55.816402 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:11:55.816402 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:11:55.816402 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:11:55.816402 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:11:55.816402 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:11:55.816402 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:11:55.818001 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:11:55.818001 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:11:55.818001 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:11:55.818001 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:11:55.818001 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:11:55.818001 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:11:55.818001 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:11:55.818001 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:11:55.818001 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:11:55.818001 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:11:55.818001 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:11:55.818001 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:11:55.818001 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 144/299] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:11:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:11:55.821181 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9209c333-d502-4dcc-928a-ba7b1786c934 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:55.831880 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe5ab78-c1d3-49c4-9e32-28265b294063 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:55.906142 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0296472-49f1-4a06-85cc-483229068bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:55.906871 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0296472-49f1-4a06-85cc-483229068bbf None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:55.907156 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0296472-49f1-4a06-85cc-483229068bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:55.907438 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0296472-49f1-4a06-85cc-483229068bbf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:55.909173 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0296472-49f1-4a06-85cc-483229068bbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:55.911087 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0296472-49f1-4a06-85cc-483229068bbf None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:55.917325 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 156/300] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:11:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:55.917325 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c226fe8-b679-418d-917c-8f15b27dcaa2 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:55.917325 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c226fe8-b679-418d-917c-8f15b27dcaa2 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:55.917325 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c226fe8-b679-418d-917c-8f15b27dcaa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:55.945593 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-4c226fe8-b679-418d-917c-8f15b27dcaa2 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:11:55.947076 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4c226fe8-b679-418d-917c-8f15b27dcaa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:11:55.993620 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:56.043434 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9209c333-d502-4dcc-928a-ba7b1786c934 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:56.059213 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 157/301] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:11:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:11:56.062878 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c226fe8-b679-418d-917c-8f15b27dcaa2 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:56.090999 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237bb0a3-81d8-4618-bbbf-b29259499adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:56.092003 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237bb0a3-81d8-4618-bbbf-b29259499adb None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:56.092443 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237bb0a3-81d8-4618-bbbf-b29259499adb None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:56.092871 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237bb0a3-81d8-4618-bbbf-b29259499adb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:56.093928 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237bb0a3-81d8-4618-bbbf-b29259499adb None admin] RBAC: Adding query filter params (role.id=da6d4a0ab8c04809ba120170fb4ae8a9, scope.project.id=ee325f0b1b08455686f9e0a696ca57c8, user.id=99487f792aae445d847c795ec9018376) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:56.094651 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237bb0a3-81d8-4618-bbbf-b29259499adb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:56.096824 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237bb0a3-81d8-4618-bbbf-b29259499adb None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:56.104577 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 145/302] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:11:55 2022] GET /identity/v3/role_assignments?user.id=99487f792aae445d847c795ec9018376&scope.project.id=ee325f0b1b08455686f9e0a696ca57c8&role.id=da6d4a0ab8c04809ba120170fb4ae8a9 => generated 608 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:56.105796 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237bb0a3-81d8-4618-bbbf-b29259499adb None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:56.187481 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:56.274254 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c226fe8-b679-418d-917c-8f15b27dcaa2 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:56.325945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237bb0a3-81d8-4618-bbbf-b29259499adb None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:56.354225 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9ce934-01da-4ea5-bce1-1d3b41e52582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:56.355501 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ce934-01da-4ea5-bce1-1d3b41e52582 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:56.356716 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ce934-01da-4ea5-bce1-1d3b41e52582 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:56.357316 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ce934-01da-4ea5-bce1-1d3b41e52582 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:56.358730 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9ce934-01da-4ea5-bce1-1d3b41e52582 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:56.359220 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9ce934-01da-4ea5-bce1-1d3b41e52582 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:56.361667 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9ce934-01da-4ea5-bce1-1d3b41e52582 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:56.377267 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 158/303] 10.222.0.87 () {56 vars in 1101 bytes} [Tue Jun 21 18:11:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:56.378275 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9ce934-01da-4ea5-bce1-1d3b41e52582 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:56.406431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bfedc8-3bf1-4654-b8a3-c537657c883e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:56.407872 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bfedc8-3bf1-4654-b8a3-c537657c883e None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:56.408883 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bfedc8-3bf1-4654-b8a3-c537657c883e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:56.409401 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bfedc8-3bf1-4654-b8a3-c537657c883e None admin] PATH_INFO: `/v3/groups` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:56.411578 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bfedc8-3bf1-4654-b8a3-c537657c883e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:56.415028 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bfedc8-3bf1-4654-b8a3-c537657c883e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:56.433157 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:56.449807 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 146/304] 10.222.0.87 () {60 vars in 1134 bytes} [Tue Jun 21 18:11:56 2022] POST /identity/v3/groups => generated 229 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:11:56.456750 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bfedc8-3bf1-4654-b8a3-c537657c883e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:56.646315 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9ce934-01da-4ea5-bce1-1d3b41e52582 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:56.688560 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bfedc8-3bf1-4654-b8a3-c537657c883e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:56.699342 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:56.707439 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7857d5d6-70b0-4ff4-8846-e0164b9af97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:56.709321 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7857d5d6-70b0-4ff4-8846-e0164b9af97b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:56.710022 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7857d5d6-70b0-4ff4-8846-e0164b9af97b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:56.710562 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7857d5d6-70b0-4ff4-8846-e0164b9af97b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:56.729606 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7857d5d6-70b0-4ff4-8846-e0164b9af97b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:11:56.741442 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-7857d5d6-70b0-4ff4-8846-e0164b9af97b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:11:56.741442 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:11:56.741442 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:11:56.741442 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:11:56.741442 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:11:56.741442 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:11:56.741442 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:11:56.741442 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:11:56.741442 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:11:56.741442 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:11:56.741442 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:11:56.741442 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:11:56.741442 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:11:56.741442 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:11:56.741442 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:11:56.741442 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:11:56.743360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:11:56.743360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:11:56.743360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:11:56.743360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:11:56.743360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:11:56.743360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:11:56.743360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:11:56.743360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:11:56.743360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:11:56.743360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:11:56.743360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:11:56.743360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:11:56.743360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:11:56.743360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:11:56.743360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:11:56.743360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:11:56.744146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:11:56.744146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:11:56.744146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:11:56.744146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:11:56.744146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:11:56.744146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:11:56.744146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:11:56.744146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:11:56.744146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:11:56.744146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:11:56.744146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:11:56.744146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:11:56.744146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 159/305] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:11:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:11:56.744146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425ef89-6322-4e7f-9738-29436907603d None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:56.744892 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425ef89-6322-4e7f-9738-29436907603d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:56.744892 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425ef89-6322-4e7f-9738-29436907603d None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:56.744892 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 160/306] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:11:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:11:56.749873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425ef89-6322-4e7f-9738-29436907603d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:56.759355 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bee2fc3-361a-4dd7-afdd-61a7160dfaf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:56.761406 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee2fc3-361a-4dd7-afdd-61a7160dfaf0 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:56.761866 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee2fc3-361a-4dd7-afdd-61a7160dfaf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:56.762340 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee2fc3-361a-4dd7-afdd-61a7160dfaf0 None admin] PATH_INFO: `/v3/users/a5f2f921ad744fbbb3c102f92dda5360` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:56.790094 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bee2fc3-361a-4dd7-afdd-61a7160dfaf0 None admin] RBAC: Authorizing `identity:get_user(user_id=a5f2f921ad744fbbb3c102f92dda5360)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:56.792863 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bee2fc3-361a-4dd7-afdd-61a7160dfaf0 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:56.795141 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 147/307] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:11:56 2022] GET /identity/v3/users/a5f2f921ad744fbbb3c102f92dda5360 => generated 293 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:56.797250 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41906feb-90ba-449a-a151-d574054e5125 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:56.797781 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41906feb-90ba-449a-a151-d574054e5125 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:56.798287 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41906feb-90ba-449a-a151-d574054e5125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:56.829364 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-41906feb-90ba-449a-a151-d574054e5125 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:11:56.832440 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-41906feb-90ba-449a-a151-d574054e5125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:11:56.931153 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:56.972208 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425ef89-6322-4e7f-9738-29436907603d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:56.993671 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 148/308] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:11:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:11:56.993671 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41906feb-90ba-449a-a151-d574054e5125 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:57.043711 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ee3f98-05a5-46c2-92d0-25f80419ca53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:57.044859 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ee3f98-05a5-46c2-92d0-25f80419ca53 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:57.045371 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ee3f98-05a5-46c2-92d0-25f80419ca53 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:57.045926 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ee3f98-05a5-46c2-92d0-25f80419ca53 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:57.047423 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ee3f98-05a5-46c2-92d0-25f80419ca53 None admin] RBAC: Adding query filter params (name=member) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:57.047915 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ee3f98-05a5-46c2-92d0-25f80419ca53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:57.050892 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ee3f98-05a5-46c2-92d0-25f80419ca53 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:57.072111 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 161/309] 10.222.0.87 () {56 vars in 1101 bytes} [Tue Jun 21 18:11:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:57.072672 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ee3f98-05a5-46c2-92d0-25f80419ca53 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:57.144794 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:57.159847 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41906feb-90ba-449a-a151-d574054e5125 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:57.234796 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65fbc35-5279-4f25-a648-5b4659e2dccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:57.235725 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65fbc35-5279-4f25-a648-5b4659e2dccd None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:57.236321 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65fbc35-5279-4f25-a648-5b4659e2dccd None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:57.236737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65fbc35-5279-4f25-a648-5b4659e2dccd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:57.238388 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65fbc35-5279-4f25-a648-5b4659e2dccd None admin] RBAC: Adding query filter params (role.id=5efb4931668c4edf87773a1f5d47f005, user.id=a5f2f921ad744fbbb3c102f92dda5360, scope.system=all) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:57.239053 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65fbc35-5279-4f25-a648-5b4659e2dccd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:57.240836 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65fbc35-5279-4f25-a648-5b4659e2dccd None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:57.252456 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 149/310] 10.222.0.87 () {56 vars in 1308 bytes} [Tue Jun 21 18:11:56 2022] GET /identity/v3/role_assignments?user.id=a5f2f921ad744fbbb3c102f92dda5360&scope.system=all&role.id=5efb4931668c4edf87773a1f5d47f005 => generated 226 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:57.254066 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65fbc35-5279-4f25-a648-5b4659e2dccd None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:57.288508 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ee3f98-05a5-46c2-92d0-25f80419ca53 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:57.386781 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64c2d4b-4c8a-47e5-969b-cfc4b824368a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:57.387720 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64c2d4b-4c8a-47e5-969b-cfc4b824368a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:57.388012 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64c2d4b-4c8a-47e5-969b-cfc4b824368a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:57.388498 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64c2d4b-4c8a-47e5-969b-cfc4b824368a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:57.416642 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64c2d4b-4c8a-47e5-969b-cfc4b824368a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:11:57.433458 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-d64c2d4b-4c8a-47e5-969b-cfc4b824368a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:11:57.433458 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:11:57.433458 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:11:57.433458 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:11:57.433458 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:11:57.433458 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:11:57.433458 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:11:57.433458 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:11:57.433458 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:11:57.433458 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:11:57.433458 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:11:57.433458 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:11:57.433458 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:11:57.433458 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:11:57.433458 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:11:57.433458 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:11:57.435296 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:11:57.435296 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:11:57.435296 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:11:57.435296 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:11:57.435296 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:11:57.435296 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:11:57.435296 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:11:57.435296 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:11:57.435296 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:11:57.435296 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:11:57.435296 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:11:57.435296 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:11:57.435296 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:11:57.435296 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:11:57.435296 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:11:57.435296 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:11:57.437801 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:11:57.437801 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:11:57.437801 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:11:57.437801 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:11:57.437801 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:11:57.437801 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:11:57.437801 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:11:57.437801 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:11:57.437801 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:11:57.437801 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:11:57.437801 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:11:57.437801 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:11:57.437801 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 162/311] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:11:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:11:57.445679 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64c2d4b-4c8a-47e5-969b-cfc4b824368a None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:57.469861 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:57.514933 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65fbc35-5279-4f25-a648-5b4659e2dccd None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:57.609881 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc91cfe-2454-44f9-bc11-9958d5259684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:57.613351 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc91cfe-2454-44f9-bc11-9958d5259684 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:57.614675 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc91cfe-2454-44f9-bc11-9958d5259684 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:57.615410 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc91cfe-2454-44f9-bc11-9958d5259684 None admin] PATH_INFO: `/v3/users/0de5ac62220f43ee93507be46407afed` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:57.618572 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc91cfe-2454-44f9-bc11-9958d5259684 None admin] RBAC: Authorizing `identity:get_user(user_id=0de5ac62220f43ee93507be46407afed)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:57.623203 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc91cfe-2454-44f9-bc11-9958d5259684 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:57.627359 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 150/312] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:11:57 2022] GET /identity/v3/users/0de5ac62220f43ee93507be46407afed => generated 293 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:57.640637 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc91cfe-2454-44f9-bc11-9958d5259684 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:57.698150 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64c2d4b-4c8a-47e5-969b-cfc4b824368a None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:57.704664 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:57.774437 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc5346b-5a6f-4393-b988-a5efe1d506d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:57.775933 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5346b-5a6f-4393-b988-a5efe1d506d4 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:57.776667 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5346b-5a6f-4393-b988-a5efe1d506d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:57.777373 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5346b-5a6f-4393-b988-a5efe1d506d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:57.779256 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc5346b-5a6f-4393-b988-a5efe1d506d4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:57.780114 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc5346b-5a6f-4393-b988-a5efe1d506d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:57.783285 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc5346b-5a6f-4393-b988-a5efe1d506d4 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:57.808443 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 163/313] 10.222.0.87 () {56 vars in 1101 bytes} [Tue Jun 21 18:11:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:57.811837 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc5346b-5a6f-4393-b988-a5efe1d506d4 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:57.848167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc91cfe-2454-44f9-bc11-9958d5259684 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:57.916830 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d33e498-a72d-4529-a1fe-4988d2f6d3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:57.918153 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33e498-a72d-4529-a1fe-4988d2f6d3ce None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:57.918718 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33e498-a72d-4529-a1fe-4988d2f6d3ce None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:57.919166 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33e498-a72d-4529-a1fe-4988d2f6d3ce None admin] PATH_INFO: `/v3/system/users/0de5ac62220f43ee93507be46407afed/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:57.923039 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d33e498-a72d-4529-a1fe-4988d2f6d3ce None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0de5ac62220f43ee93507be46407afed, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:57.926434 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d33e498-a72d-4529-a1fe-4988d2f6d3ce None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:57.945611 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 151/314] 10.222.0.87 () {58 vars in 1334 bytes} [Tue Jun 21 18:11:57 2022] PUT /identity/v3/system/users/0de5ac62220f43ee93507be46407afed/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:11:57.964491 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:58.050362 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc5346b-5a6f-4393-b988-a5efe1d506d4 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:58.137540 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230fe476-078d-4052-9612-43f98554e1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:58.143112 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230fe476-078d-4052-9612-43f98554e1d4 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:58.143992 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230fe476-078d-4052-9612-43f98554e1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:58.144769 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230fe476-078d-4052-9612-43f98554e1d4 None admin] PATH_INFO: `/v3/users/ca6ab7a1cd61437288936f3c5c0f2494` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:58.148046 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230fe476-078d-4052-9612-43f98554e1d4 None admin] RBAC: Authorizing `identity:get_user(user_id=ca6ab7a1cd61437288936f3c5c0f2494)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:58.158773 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230fe476-078d-4052-9612-43f98554e1d4 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:58.164274 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 164/315] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:11:57 2022] GET /identity/v3/users/ca6ab7a1cd61437288936f3c5c0f2494 => generated 297 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:11:58.174901 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d33e498-a72d-4529-a1fe-4988d2f6d3ce None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:58.375300 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:58.398783 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d33e498-a72d-4529-a1fe-4988d2f6d3ce None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:58.483613 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e8eb4-c48c-4c0a-a81a-fc220cbd345b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:58.483933 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e8eb4-c48c-4c0a-a81a-fc220cbd345b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:58.484339 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e8eb4-c48c-4c0a-a81a-fc220cbd345b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:58.489494 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e8eb4-c48c-4c0a-a81a-fc220cbd345b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:58.631579 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2e8eb4-c48c-4c0a-a81a-fc220cbd345b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:58.634274 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2e8eb4-c48c-4c0a-a81a-fc220cbd345b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:58.817135 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:11:58.838445 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 152/316] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:11:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 665 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:11:58.855500 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230fe476-078d-4052-9612-43f98554e1d4 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:59.066663 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230fe476-078d-4052-9612-43f98554e1d4 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:59.135254 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac54c6d7-ea0a-4628-bdc4-9c85ca8a8b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:59.136820 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54c6d7-ea0a-4628-bdc4-9c85ca8a8b6a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:59.137305 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54c6d7-ea0a-4628-bdc4-9c85ca8a8b6a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:59.137829 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54c6d7-ea0a-4628-bdc4-9c85ca8a8b6a None admin] PATH_INFO: `/v3/projects/ee325f0b1b08455686f9e0a696ca57c8` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:59.140139 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac54c6d7-ea0a-4628-bdc4-9c85ca8a8b6a None admin] RBAC: Authorizing `identity:get_project(project_id=ee325f0b1b08455686f9e0a696ca57c8)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:59.143118 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac54c6d7-ea0a-4628-bdc4-9c85ca8a8b6a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:59.146647 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 165/317] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:11:58 2022] GET /identity/v3/projects/ee325f0b1b08455686f9e0a696ca57c8 => generated 302 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:59.157284 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e8eb4-c48c-4c0a-a81a-fc220cbd345b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:11:59.360597 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e8eb4-c48c-4c0a-a81a-fc220cbd345b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:11:59.426417 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac437ba6-24ce-4618-a39a-25db85be0429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:11:59.428031 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac437ba6-24ce-4618-a39a-25db85be0429 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:59.428845 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac437ba6-24ce-4618-a39a-25db85be0429 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:59.429677 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac437ba6-24ce-4618-a39a-25db85be0429 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:59.431732 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac437ba6-24ce-4618-a39a-25db85be0429 None admin] RBAC: Adding query filter params (role.id=5efb4931668c4edf87773a1f5d47f005, scope.project.id=ee325f0b1b08455686f9e0a696ca57c8, user.id=ca6ab7a1cd61437288936f3c5c0f2494) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:11:59.432683 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac437ba6-24ce-4618-a39a-25db85be0429 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:11:59.435705 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac437ba6-24ce-4618-a39a-25db85be0429 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:11:59.448345 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 153/318] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:11:59 2022] GET /identity/v3/role_assignments?user.id=ca6ab7a1cd61437288936f3c5c0f2494&scope.project.id=ee325f0b1b08455686f9e0a696ca57c8&role.id=5efb4931668c4edf87773a1f5d47f005 => generated 608 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:11:59.799264 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae5d1fb-aa4f-43bc-9d8d-5e4f21df370c None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:59.800278 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae5d1fb-aa4f-43bc-9d8d-5e4f21df370c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:59.800814 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae5d1fb-aa4f-43bc-9d8d-5e4f21df370c None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:59.803460 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 166/319] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:11:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:11:59.811153 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1021dc5-ed27-4c75-b767-d07e7a7d32cb None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:11:59.812011 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1021dc5-ed27-4c75-b767-d07e7a7d32cb None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:11:59.812796 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1021dc5-ed27-4c75-b767-d07e7a7d32cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:11:59.850918 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b1021dc5-ed27-4c75-b767-d07e7a7d32cb None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:11:59.852953 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b1021dc5-ed27-4c75-b767-d07e7a7d32cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:11:59.965009 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:00.009705 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 154/320] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:11:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:12:00.046692 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae5d1fb-aa4f-43bc-9d8d-5e4f21df370c None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:00.064029 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:00.185599 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae5d1fb-aa4f-43bc-9d8d-5e4f21df370c None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:00.228203 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196eb85e-bd43-4fa2-9a01-b27152ebb198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:00.229103 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196eb85e-bd43-4fa2-9a01-b27152ebb198 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:00.229404 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196eb85e-bd43-4fa2-9a01-b27152ebb198 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:00.229673 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196eb85e-bd43-4fa2-9a01-b27152ebb198 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:00.231035 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196eb85e-bd43-4fa2-9a01-b27152ebb198 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:00.232999 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196eb85e-bd43-4fa2-9a01-b27152ebb198 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:00.235267 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 167/321] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:12:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:00.243096 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1021dc5-ed27-4c75-b767-d07e7a7d32cb None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:00.456834 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1021dc5-ed27-4c75-b767-d07e7a7d32cb None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:00.535925 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7130cec1-2f34-425e-8254-5d9122f52756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:00.537320 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130cec1-2f34-425e-8254-5d9122f52756 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:00.537912 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130cec1-2f34-425e-8254-5d9122f52756 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:00.539008 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130cec1-2f34-425e-8254-5d9122f52756 None admin] PATH_INFO: `/v3/groups` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:00.540002 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7130cec1-2f34-425e-8254-5d9122f52756 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:00.543060 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7130cec1-2f34-425e-8254-5d9122f52756 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:00.576697 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 155/322] 10.222.0.87 () {60 vars in 1134 bytes} [Tue Jun 21 18:12:00 2022] POST /identity/v3/groups => generated 226 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:12:00.594824 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a0f2c-6f94-4ab7-b4fd-813bc33eb1e0 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:00.595834 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a0f2c-6f94-4ab7-b4fd-813bc33eb1e0 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:00.595834 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a0f2c-6f94-4ab7-b4fd-813bc33eb1e0 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:00.599030 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 168/323] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:12:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:12:00.609940 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f167b9-011f-42bf-9427-d84b771ecdd6 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:00.610450 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f167b9-011f-42bf-9427-d84b771ecdd6 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:00.611115 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f167b9-011f-42bf-9427-d84b771ecdd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:00.649401 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-59f167b9-011f-42bf-9427-d84b771ecdd6 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:12:00.652415 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-59f167b9-011f-42bf-9427-d84b771ecdd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:12:00.756234 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:00.824072 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 156/324] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:12:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:12:00.865498 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835a0f2c-6f94-4ab7-b4fd-813bc33eb1e0 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:01.039520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:01.092774 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835a0f2c-6f94-4ab7-b4fd-813bc33eb1e0 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:01.164685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c63db-8700-4b7a-9392-b163dc1771ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:01.166163 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c63db-8700-4b7a-9392-b163dc1771ce None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:01.166665 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c63db-8700-4b7a-9392-b163dc1771ce None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:01.167138 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c63db-8700-4b7a-9392-b163dc1771ce None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:01.192993 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389c63db-8700-4b7a-9392-b163dc1771ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:12:01.209159 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-389c63db-8700-4b7a-9392-b163dc1771ce None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:12:01.209159 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:01.209159 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:01.209159 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:01.209159 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:01.209159 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:01.209159 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:01.209159 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:01.209159 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:01.209159 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:01.209159 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:01.209159 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:01.209159 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:12:01.209159 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:12:01.209159 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:12:01.209159 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:12:01.210997 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:12:01.210997 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:12:01.210997 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:12:01.210997 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:12:01.210997 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:01.210997 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:01.210997 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:01.210997 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:01.210997 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:01.210997 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:01.210997 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:01.210997 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:01.210997 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:01.210997 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:01.210997 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:01.210997 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:01.212516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:01.212516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:01.212516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:01.212516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:01.212516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:12:01.212516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:12:01.212516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:12:01.212516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:12:01.212516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:12:01.212516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:12:01.212516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:12:01.212516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:12:01.212516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 169/325] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:12:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:01.221918 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f167b9-011f-42bf-9427-d84b771ecdd6 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:01.244781 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab8c97-dcb9-4b17-8b0c-c21200c31688 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:01.245225 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab8c97-dcb9-4b17-8b0c-c21200c31688 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:01.245709 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab8c97-dcb9-4b17-8b0c-c21200c31688 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:01.248440 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 170/326] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:12:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:12:01.257357 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86587c-dfe6-4fe3-9e06-861876e380d9 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:01.257876 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86587c-dfe6-4fe3-9e06-861876e380d9 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:01.258329 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86587c-dfe6-4fe3-9e06-861876e380d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:01.286930 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-ec86587c-dfe6-4fe3-9e06-861876e380d9 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:12:01.288854 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ec86587c-dfe6-4fe3-9e06-861876e380d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:12:01.386267 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 171/327] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:12:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:12:01.432061 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec86587c-dfe6-4fe3-9e06-861876e380d9 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:01.473926 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f167b9-011f-42bf-9427-d84b771ecdd6 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:01.533074 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3be2821-1310-4105-b22f-05eaa2acb4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:01.534352 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be2821-1310-4105-b22f-05eaa2acb4d2 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:01.534819 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be2821-1310-4105-b22f-05eaa2acb4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:01.535255 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be2821-1310-4105-b22f-05eaa2acb4d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:01.536461 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3be2821-1310-4105-b22f-05eaa2acb4d2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:01.536937 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3be2821-1310-4105-b22f-05eaa2acb4d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:01.539392 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3be2821-1310-4105-b22f-05eaa2acb4d2 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:01.544332 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:01.555270 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 157/328] 10.222.0.87 () {56 vars in 1101 bytes} [Tue Jun 21 18:12:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:01.571181 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3be2821-1310-4105-b22f-05eaa2acb4d2 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:01.645321 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec86587c-dfe6-4fe3-9e06-861876e380d9 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:01.646244 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:01.709820 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74881176-93b7-458d-bf2f-a42ae06d172f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:01.711455 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74881176-93b7-458d-bf2f-a42ae06d172f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:01.711979 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74881176-93b7-458d-bf2f-a42ae06d172f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:01.712499 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74881176-93b7-458d-bf2f-a42ae06d172f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:01.744827 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74881176-93b7-458d-bf2f-a42ae06d172f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:12:01.763877 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-74881176-93b7-458d-bf2f-a42ae06d172f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:12:01.763877 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:01.763877 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:01.763877 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:01.763877 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:01.763877 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:01.763877 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:01.763877 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:01.763877 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:01.763877 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:01.763877 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:01.763877 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:01.763877 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:12:01.763877 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:12:01.763877 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:12:01.763877 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:12:01.765795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:12:01.765795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:12:01.765795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:12:01.765795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:12:01.765795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:01.765795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:01.765795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:01.765795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:01.765795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:01.765795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:01.765795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:01.765795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:01.765795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:01.765795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:01.765795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:01.765795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:01.768373 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:01.768373 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:01.768373 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:01.768373 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:01.768373 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:12:01.768373 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:12:01.768373 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:12:01.768373 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:12:01.768373 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:12:01.768373 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:12:01.768373 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:12:01.768373 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:12:01.768373 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 172/329] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:12:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:12:01.776184 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74881176-93b7-458d-bf2f-a42ae06d172f None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:01.823772 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3be2821-1310-4105-b22f-05eaa2acb4d2 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:01.918032 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10873efd-2651-4e1e-8c80-227b20261180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:01.920547 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10873efd-2651-4e1e-8c80-227b20261180 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:01.921160 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10873efd-2651-4e1e-8c80-227b20261180 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:01.922680 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10873efd-2651-4e1e-8c80-227b20261180 None admin] PATH_INFO: `/v3/users/a5f2f921ad744fbbb3c102f92dda5360` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:01.925591 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10873efd-2651-4e1e-8c80-227b20261180 None admin] RBAC: Authorizing `identity:get_user(user_id=a5f2f921ad744fbbb3c102f92dda5360)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:01.930173 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10873efd-2651-4e1e-8c80-227b20261180 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:01.935387 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 158/330] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:12:01 2022] GET /identity/v3/users/a5f2f921ad744fbbb3c102f92dda5360 => generated 293 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:01.950258 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10873efd-2651-4e1e-8c80-227b20261180 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:02.049186 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74881176-93b7-458d-bf2f-a42ae06d172f None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:02.108135 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:02.133704 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c51cdb3-0947-437b-9026-8472feb7d74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:02.134871 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51cdb3-0947-437b-9026-8472feb7d74f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:02.135356 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51cdb3-0947-437b-9026-8472feb7d74f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:02.135836 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51cdb3-0947-437b-9026-8472feb7d74f None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:02.137245 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c51cdb3-0947-437b-9026-8472feb7d74f None admin] RBAC: Adding query filter params (name=member) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:02.137824 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c51cdb3-0947-437b-9026-8472feb7d74f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:02.142143 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c51cdb3-0947-437b-9026-8472feb7d74f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:02.163019 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 173/331] 10.222.0.87 () {56 vars in 1101 bytes} [Tue Jun 21 18:12:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:12:02.179964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c51cdb3-0947-437b-9026-8472feb7d74f None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:02.312779 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10873efd-2651-4e1e-8c80-227b20261180 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:02.322126 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:02.373687 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efce09e-b12f-4a1b-ae75-c14595773ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:02.374996 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efce09e-b12f-4a1b-ae75-c14595773ecc None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:02.375539 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efce09e-b12f-4a1b-ae75-c14595773ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:02.376082 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efce09e-b12f-4a1b-ae75-c14595773ecc None admin] PATH_INFO: `/v3/system/users/a5f2f921ad744fbbb3c102f92dda5360/roles/5efb4931668c4edf87773a1f5d47f005` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:02.380183 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efce09e-b12f-4a1b-ae75-c14595773ecc None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a5f2f921ad744fbbb3c102f92dda5360, role_id=5efb4931668c4edf87773a1f5d47f005)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:02.382078 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efce09e-b12f-4a1b-ae75-c14595773ecc None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:02.394923 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 159/332] 10.222.0.87 () {58 vars in 1334 bytes} [Tue Jun 21 18:12:01 2022] PUT /identity/v3/system/users/a5f2f921ad744fbbb3c102f92dda5360/roles/5efb4931668c4edf87773a1f5d47f005 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:12:02.469908 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c51cdb3-0947-437b-9026-8472feb7d74f None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:02.562366 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef086d2-01e5-4dc8-bc30-9709c1b4b252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:02.564525 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef086d2-01e5-4dc8-bc30-9709c1b4b252 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:02.565027 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef086d2-01e5-4dc8-bc30-9709c1b4b252 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:02.565504 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef086d2-01e5-4dc8-bc30-9709c1b4b252 None admin] PATH_INFO: `/v3/users/0de5ac62220f43ee93507be46407afed` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:02.568096 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef086d2-01e5-4dc8-bc30-9709c1b4b252 None admin] RBAC: Authorizing `identity:get_user(user_id=0de5ac62220f43ee93507be46407afed)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:02.570966 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef086d2-01e5-4dc8-bc30-9709c1b4b252 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:02.574300 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 174/333] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:12:02 2022] GET /identity/v3/users/0de5ac62220f43ee93507be46407afed => generated 293 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:02.585074 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efce09e-b12f-4a1b-ae75-c14595773ecc None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:02.828965 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efce09e-b12f-4a1b-ae75-c14595773ecc None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:02.916609 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b5204-23cb-4a84-9022-a54b93bc69ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:02.917823 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b5204-23cb-4a84-9022-a54b93bc69ac None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:02.918304 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b5204-23cb-4a84-9022-a54b93bc69ac None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:02.918787 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b5204-23cb-4a84-9022-a54b93bc69ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:02.920533 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7b5204-23cb-4a84-9022-a54b93bc69ac None admin] RBAC: Adding query filter params (role.id=da6d4a0ab8c04809ba120170fb4ae8a9, user.id=0de5ac62220f43ee93507be46407afed, scope.system=all) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:02.921093 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7b5204-23cb-4a84-9022-a54b93bc69ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:02.924170 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7b5204-23cb-4a84-9022-a54b93bc69ac None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:02.953755 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 160/334] 10.222.0.87 () {56 vars in 1308 bytes} [Tue Jun 21 18:12:02 2022] GET /identity/v3/role_assignments?user.id=0de5ac62220f43ee93507be46407afed&scope.system=all&role.id=da6d4a0ab8c04809ba120170fb4ae8a9 => generated 510 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:04.960249 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cafb27-7267-4183-a2a0-403621359f63 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:04.961100 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cafb27-7267-4183-a2a0-403621359f63 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:04.961275 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cafb27-7267-4183-a2a0-403621359f63 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:04.963995 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 175/335] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:12:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:12:04.973003 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236e031-98f3-415e-a578-db480e94a580 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:04.974621 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236e031-98f3-415e-a578-db480e94a580 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:04.975425 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236e031-98f3-415e-a578-db480e94a580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:05.017156 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d236e031-98f3-415e-a578-db480e94a580 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:12:05.019902 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d236e031-98f3-415e-a578-db480e94a580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:12:05.084308 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:05.206344 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 161/336] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:12:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:12:05.285214 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cafb27-7267-4183-a2a0-403621359f63 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:05.420999 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:05.551760 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cafb27-7267-4183-a2a0-403621359f63 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:05.638096 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab99fdd9-d556-4bf8-8b33-e5624b12aa71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:05.639513 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab99fdd9-d556-4bf8-8b33-e5624b12aa71 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:05.639972 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab99fdd9-d556-4bf8-8b33-e5624b12aa71 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:05.640458 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab99fdd9-d556-4bf8-8b33-e5624b12aa71 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:05.671668 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab99fdd9-d556-4bf8-8b33-e5624b12aa71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:12:05.686697 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-ab99fdd9-d556-4bf8-8b33-e5624b12aa71 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:12:05.686697 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:05.686697 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:05.686697 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:05.686697 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:05.686697 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:05.686697 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:05.686697 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:05.686697 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:05.686697 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:05.686697 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:05.686697 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:05.686697 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:12:05.686697 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:12:05.686697 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:12:05.686697 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:12:05.688455 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:12:05.688455 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:12:05.688455 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:12:05.688455 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:12:05.688455 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:05.688455 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:05.688455 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:05.688455 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:05.688455 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:05.688455 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:05.688455 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:05.688455 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:05.688455 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:05.688455 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:05.688455 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:05.688455 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:05.690786 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:05.690786 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:05.690786 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:05.690786 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:05.690786 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:12:05.690786 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:12:05.690786 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:12:05.690786 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:12:05.690786 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:12:05.690786 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:12:05.690786 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:12:05.690786 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:12:05.690786 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 176/337] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:12:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 406 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:12:05.700239 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236e031-98f3-415e-a578-db480e94a580 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:05.999575 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236e031-98f3-415e-a578-db480e94a580 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:06.087180 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f021e268-60b0-4ba4-88a6-c4f66d4feb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:06.088459 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f021e268-60b0-4ba4-88a6-c4f66d4feb0c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:06.089044 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f021e268-60b0-4ba4-88a6-c4f66d4feb0c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:06.089609 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f021e268-60b0-4ba4-88a6-c4f66d4feb0c None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:06.091383 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f021e268-60b0-4ba4-88a6-c4f66d4feb0c None admin] RBAC: Adding query filter params (name=member) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:06.092002 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f021e268-60b0-4ba4-88a6-c4f66d4feb0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:06.094947 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f021e268-60b0-4ba4-88a6-c4f66d4feb0c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:06.119022 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 162/338] 10.222.0.87 () {56 vars in 1101 bytes} [Tue Jun 21 18:12:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 420 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:06.128249 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab99fdd9-d556-4bf8-8b33-e5624b12aa71 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:06.449254 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab99fdd9-d556-4bf8-8b33-e5624b12aa71 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:06.494400 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:06.542760 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e41d0-e7d3-4aed-89ee-2f9adb9e4e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:06.544346 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e41d0-e7d3-4aed-89ee-2f9adb9e4e98 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:06.545093 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e41d0-e7d3-4aed-89ee-2f9adb9e4e98 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:06.545738 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e41d0-e7d3-4aed-89ee-2f9adb9e4e98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:06.685973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1e41d0-e7d3-4aed-89ee-2f9adb9e4e98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:06.689145 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1e41d0-e7d3-4aed-89ee-2f9adb9e4e98 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:06.700315 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7361f7-c903-4f7a-be25-3378c33d7604 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:06.701238 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7361f7-c903-4f7a-be25-3378c33d7604 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:06.702173 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7361f7-c903-4f7a-be25-3378c33d7604 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:06.705758 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 163/339] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:12:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:12:06.753053 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6832ca4d-167b-4e6b-bff9-69e53e52d338 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:06.754076 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6832ca4d-167b-4e6b-bff9-69e53e52d338 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:06.755015 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6832ca4d-167b-4e6b-bff9-69e53e52d338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:06.796096 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-6832ca4d-167b-4e6b-bff9-69e53e52d338 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:12:06.799454 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6832ca4d-167b-4e6b-bff9-69e53e52d338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:12:06.868312 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:06.956143 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 177/340] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:12:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 829 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:12:06.966663 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e41d0-e7d3-4aed-89ee-2f9adb9e4e98 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:07.002419 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:07.081846 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 164/341] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:12:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:12:07.238614 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e41d0-e7d3-4aed-89ee-2f9adb9e4e98 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:07.294940 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6832ca4d-167b-4e6b-bff9-69e53e52d338 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:07.316377 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5064ad64-fdbe-428e-ae4b-ab0980eb47fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:07.321683 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5064ad64-fdbe-428e-ae4b-ab0980eb47fe None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:07.321683 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5064ad64-fdbe-428e-ae4b-ab0980eb47fe None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:07.321683 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5064ad64-fdbe-428e-ae4b-ab0980eb47fe None admin] PATH_INFO: `/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:07.326352 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5064ad64-fdbe-428e-ae4b-ab0980eb47fe None admin] RBAC: Authorizing `identity:get_project(project_id=48caa0e9d7fe40d28e99abd1a7a6896f)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:07.329307 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5064ad64-fdbe-428e-ae4b-ab0980eb47fe None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:07.333684 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 178/342] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:12:06 2022] GET /identity/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f => generated 298 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:12:07.341684 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5064ad64-fdbe-428e-ae4b-ab0980eb47fe None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:07.497621 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6832ca4d-167b-4e6b-bff9-69e53e52d338 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:07.556511 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafcd887-7c4e-4bf0-8200-3277059b8728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:07.558905 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafcd887-7c4e-4bf0-8200-3277059b8728 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:07.559785 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafcd887-7c4e-4bf0-8200-3277059b8728 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:07.560590 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafcd887-7c4e-4bf0-8200-3277059b8728 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:07.584715 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafcd887-7c4e-4bf0-8200-3277059b8728 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:12:07.598842 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:07.610791 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-eafcd887-7c4e-4bf0-8200-3277059b8728 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:12:07.610791 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:07.610791 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:07.610791 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:07.610791 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:07.610791 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:07.610791 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:07.610791 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:07.610791 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:07.610791 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:07.610791 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:07.610791 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:07.610791 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:12:07.610791 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:12:07.610791 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:12:07.610791 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:12:07.612567 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:12:07.612567 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:12:07.612567 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:12:07.612567 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:12:07.612567 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:07.612567 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:07.612567 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:07.612567 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:07.612567 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:07.612567 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:07.612567 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:07.612567 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:07.612567 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:07.612567 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:07.612567 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:07.612567 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:07.614203 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:07.614203 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:07.614203 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:07.614203 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:07.614203 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:12:07.614203 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:12:07.614203 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:12:07.614203 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:12:07.614203 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:12:07.614203 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:12:07.614203 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:12:07.614203 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:12:07.620782 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 165/343] 10.222.0.87 () {56 vars in 1114 bytes} [Tue Jun 21 18:12:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:12:07.623772 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0a2be-1beb-487f-8dd9-1a92fa77033c None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:07.624655 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0a2be-1beb-487f-8dd9-1a92fa77033c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:07.625386 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0a2be-1beb-487f-8dd9-1a92fa77033c None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:07.628983 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 166/344] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:12:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:12:07.631571 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c0a2be-1beb-487f-8dd9-1a92fa77033c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:07.655107 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:07.660303 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5064ad64-fdbe-428e-ae4b-ab0980eb47fe None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:07.791734 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e1b1b3-e4b2-4286-9cbd-9871871e7244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:07.794083 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e1b1b3-e4b2-4286-9cbd-9871871e7244 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:07.796911 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e1b1b3-e4b2-4286-9cbd-9871871e7244 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:07.797794 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e1b1b3-e4b2-4286-9cbd-9871871e7244 None admin] PATH_INFO: `/v3/groups/b95be051ba524e2986ba6efc42648e76` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:07.823203 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e1b1b3-e4b2-4286-9cbd-9871871e7244 None admin] RBAC: Authorizing `identity:get_group(group_id=b95be051ba524e2986ba6efc42648e76)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:07.826140 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e1b1b3-e4b2-4286-9cbd-9871871e7244 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:07.828660 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 179/345] 10.222.0.87 () {56 vars in 1180 bytes} [Tue Jun 21 18:12:07 2022] GET /identity/v3/groups/b95be051ba524e2986ba6efc42648e76 => generated 226 bytes in 490 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:12:07.830327 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec58423-16ce-4346-962d-0515f6b76f08 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:07.830615 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec58423-16ce-4346-962d-0515f6b76f08 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:07.830894 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec58423-16ce-4346-962d-0515f6b76f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:07.853671 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-0ec58423-16ce-4346-962d-0515f6b76f08 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:12:07.853671 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-0ec58423-16ce-4346-962d-0515f6b76f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:12:07.966936 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 180/346] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:12:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:12:07.968073 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec58423-16ce-4346-962d-0515f6b76f08 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:07.978538 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c0a2be-1beb-487f-8dd9-1a92fa77033c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:08.077434 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:08.091422 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f05aef-43c2-40e9-9223-992eac7dad36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:08.093233 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f05aef-43c2-40e9-9223-992eac7dad36 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:08.093769 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f05aef-43c2-40e9-9223-992eac7dad36 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:08.094289 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f05aef-43c2-40e9-9223-992eac7dad36 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:08.095787 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f05aef-43c2-40e9-9223-992eac7dad36 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:08.096311 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f05aef-43c2-40e9-9223-992eac7dad36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:08.099042 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f05aef-43c2-40e9-9223-992eac7dad36 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:08.119845 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 167/347] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:12:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 490 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:08.120863 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec58423-16ce-4346-962d-0515f6b76f08 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:08.122294 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f05aef-43c2-40e9-9223-992eac7dad36 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:08.192041 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2888ca8e-6dde-4720-b8b1-62558e835350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:08.195720 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888ca8e-6dde-4720-b8b1-62558e835350 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:08.195720 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888ca8e-6dde-4720-b8b1-62558e835350 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:08.195720 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888ca8e-6dde-4720-b8b1-62558e835350 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:08.196430 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2888ca8e-6dde-4720-b8b1-62558e835350 None admin] RBAC: Adding query filter params (group.id=b95be051ba524e2986ba6efc42648e76, role.id=da6d4a0ab8c04809ba120170fb4ae8a9, scope.project.id=48caa0e9d7fe40d28e99abd1a7a6896f) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:08.196868 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2888ca8e-6dde-4720-b8b1-62558e835350 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:08.200727 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2888ca8e-6dde-4720-b8b1-62558e835350 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:08.220580 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 181/348] 10.222.0.87 () {56 vars in 1376 bytes} [Tue Jun 21 18:12:07 2022] GET /identity/v3/role_assignments?group.id=b95be051ba524e2986ba6efc42648e76&scope.project.id=48caa0e9d7fe40d28e99abd1a7a6896f&role.id=da6d4a0ab8c04809ba120170fb4ae8a9 => generated 260 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:08.223775 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717192e2-f979-4202-9d73-9e4a1d657bee None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:08.224745 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717192e2-f979-4202-9d73-9e4a1d657bee None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:08.225577 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717192e2-f979-4202-9d73-9e4a1d657bee None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:08.228464 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 182/349] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:12:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:12:08.230542 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717192e2-f979-4202-9d73-9e4a1d657bee None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:08.270834 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:08.329149 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f05aef-43c2-40e9-9223-992eac7dad36 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:08.379207 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdf7e56-e248-4d4a-8689-aaf70a093759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:08.380605 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf7e56-e248-4d4a-8689-aaf70a093759 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:08.381111 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf7e56-e248-4d4a-8689-aaf70a093759 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:08.381615 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf7e56-e248-4d4a-8689-aaf70a093759 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:08.395644 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdf7e56-e248-4d4a-8689-aaf70a093759 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:12:08.404502 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-7bdf7e56-e248-4d4a-8689-aaf70a093759 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:12:08.404502 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:08.404502 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:08.404502 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:08.404502 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:08.404502 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:08.404502 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:08.404502 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:08.404502 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:08.404502 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:08.404502 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:08.404502 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:08.404502 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:12:08.404502 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:12:08.404502 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:12:08.404502 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:12:08.405503 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:12:08.405503 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:12:08.405503 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:12:08.405503 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:12:08.405503 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:08.405503 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:08.405503 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:08.405503 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:08.405503 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:08.405503 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:08.405503 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:08.405503 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:08.405503 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:08.405503 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:08.405503 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:08.405503 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:08.406362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:08.406362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:08.406362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:08.406362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:08.406362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:12:08.406362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:12:08.406362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:12:08.406362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:12:08.406362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:12:08.406362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:12:08.406362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:12:08.406362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:12:08.409297 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 168/350] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:12:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:08.411419 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512689b-135f-482c-b664-bc2c96e90631 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:08.411994 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512689b-135f-482c-b664-bc2c96e90631 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:08.412472 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512689b-135f-482c-b664-bc2c96e90631 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:08.414632 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 169/351] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:12:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:12:08.416400 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ee435-f3d4-4861-9999-81b8f41cbf34 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:08.416917 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ee435-f3d4-4861-9999-81b8f41cbf34 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:08.417512 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ee435-f3d4-4861-9999-81b8f41cbf34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:08.439243 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-2b9ee435-f3d4-4861-9999-81b8f41cbf34 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:12:08.440858 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2b9ee435-f3d4-4861-9999-81b8f41cbf34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:12:08.458461 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717192e2-f979-4202-9d73-9e4a1d657bee None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:08.528122 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add3b914-0698-462d-b8a7-ef88d83794ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:08.529719 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add3b914-0698-462d-b8a7-ef88d83794ed None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:08.530604 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add3b914-0698-462d-b8a7-ef88d83794ed None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:08.531305 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add3b914-0698-462d-b8a7-ef88d83794ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:08.552750 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:08.597389 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 170/352] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:12:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:12:08.599303 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9ee435-f3d4-4861-9999-81b8f41cbf34 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:08.679921 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add3b914-0698-462d-b8a7-ef88d83794ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:08.682621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add3b914-0698-462d-b8a7-ef88d83794ed None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:08.697004 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:08.770199 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9ee435-f3d4-4861-9999-81b8f41cbf34 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:08.812765 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4271868a-88dc-4283-8513-1cc63ac8d171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:08.814070 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271868a-88dc-4283-8513-1cc63ac8d171 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:08.814539 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271868a-88dc-4283-8513-1cc63ac8d171 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:08.814971 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271868a-88dc-4283-8513-1cc63ac8d171 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:08.816152 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4271868a-88dc-4283-8513-1cc63ac8d171 None admin] RBAC: Adding query filter params (name=member) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:08.816647 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4271868a-88dc-4283-8513-1cc63ac8d171 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:08.818391 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4271868a-88dc-4283-8513-1cc63ac8d171 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:08.830057 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 171/353] 10.222.0.87 () {56 vars in 1101 bytes} [Tue Jun 21 18:12:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:08.832274 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b996647-c2bb-4960-9f61-c1982e82c5e4 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:08.832775 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b996647-c2bb-4960-9f61-c1982e82c5e4 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:08.833234 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b996647-c2bb-4960-9f61-c1982e82c5e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:08.854359 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-9b996647-c2bb-4960-9f61-c1982e82c5e4 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:12:08.855642 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9b996647-c2bb-4960-9f61-c1982e82c5e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:12:08.902169 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 183/354] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:12:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 673 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:12:08.904244 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08148a3d-6bfc-47f5-a343-587a0f2157c4 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:08.904827 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08148a3d-6bfc-47f5-a343-587a0f2157c4 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:08.905406 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08148a3d-6bfc-47f5-a343-587a0f2157c4 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:08.913664 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 184/355] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:12:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:12:08.913664 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08148a3d-6bfc-47f5-a343-587a0f2157c4 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:08.969966 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 172/356] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:12:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:12:08.970929 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b996647-c2bb-4960-9f61-c1982e82c5e4 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:08.982846 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:09.063149 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:09.093422 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08148a3d-6bfc-47f5-a343-587a0f2157c4 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:09.132280 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b996647-c2bb-4960-9f61-c1982e82c5e4 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:09.157034 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a9aeba-09f8-44d7-80e3-4af4cd22649f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:09.158589 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9aeba-09f8-44d7-80e3-4af4cd22649f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:09.158870 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9aeba-09f8-44d7-80e3-4af4cd22649f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:09.159195 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9aeba-09f8-44d7-80e3-4af4cd22649f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:09.177310 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a9aeba-09f8-44d7-80e3-4af4cd22649f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:12:09.187473 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-13a9aeba-09f8-44d7-80e3-4af4cd22649f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:12:09.187473 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:09.187473 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:09.187473 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:09.187473 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:09.187473 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:09.187473 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:09.187473 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:09.187473 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:09.187473 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:09.187473 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:09.187473 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:09.187473 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:12:09.187473 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:12:09.187473 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:12:09.187473 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:12:09.189284 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:12:09.189284 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:12:09.189284 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:12:09.189284 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:12:09.189284 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:09.189284 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:09.189284 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:09.189284 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:09.189284 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:09.189284 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:09.189284 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:09.189284 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:09.189284 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:09.189284 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:09.189284 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:09.189284 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:09.190956 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:09.190956 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:09.190956 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:09.190956 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:09.190956 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:12:09.190956 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:12:09.190956 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:12:09.190956 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:12:09.190956 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:12:09.190956 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:12:09.190956 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:12:09.190956 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:12:09.190956 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 185/357] 10.222.0.87 () {56 vars in 1114 bytes} [Tue Jun 21 18:12:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:09.190956 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a9aeba-09f8-44d7-80e3-4af4cd22649f None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:09.201922 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7383ad-b44b-444d-907f-4c966138e985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:09.203813 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7383ad-b44b-444d-907f-4c966138e985 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:09.203813 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7383ad-b44b-444d-907f-4c966138e985 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:09.204700 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7383ad-b44b-444d-907f-4c966138e985 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:09.322233 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7383ad-b44b-444d-907f-4c966138e985 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:09.325095 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7383ad-b44b-444d-907f-4c966138e985 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:09.395199 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:09.398716 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a9aeba-09f8-44d7-80e3-4af4cd22649f None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:09.437463 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:09.450166 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a273896f-fd87-4a57-ae82-200c870a9520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:09.451087 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273896f-fd87-4a57-ae82-200c870a9520 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:09.451376 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273896f-fd87-4a57-ae82-200c870a9520 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:09.451660 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273896f-fd87-4a57-ae82-200c870a9520 None admin] PATH_INFO: `/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:09.453525 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a273896f-fd87-4a57-ae82-200c870a9520 None admin] RBAC: Authorizing `identity:get_project(project_id=48caa0e9d7fe40d28e99abd1a7a6896f)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:09.455832 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a273896f-fd87-4a57-ae82-200c870a9520 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:09.458490 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 186/358] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:12:09 2022] GET /identity/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f => generated 298 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:09.460335 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7687c3ed-944f-49f8-a21c-d971b0d11935 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:09.460652 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7687c3ed-944f-49f8-a21c-d971b0d11935 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:09.461258 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7687c3ed-944f-49f8-a21c-d971b0d11935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:09.482840 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-7687c3ed-944f-49f8-a21c-d971b0d11935 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:12:09.485039 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7687c3ed-944f-49f8-a21c-d971b0d11935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:12:09.523455 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 173/359] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:12:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:12:09.529226 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7383ad-b44b-444d-907f-4c966138e985 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:09.607879 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 187/360] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:12:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:12:09.609389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7687c3ed-944f-49f8-a21c-d971b0d11935 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:09.733523 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7383ad-b44b-444d-907f-4c966138e985 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:09.803164 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e668a8f8-c27e-44d4-a466-c6d497f464c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:09.804095 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e668a8f8-c27e-44d4-a466-c6d497f464c3 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:09.804384 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e668a8f8-c27e-44d4-a466-c6d497f464c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:09.804683 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e668a8f8-c27e-44d4-a466-c6d497f464c3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:09.807838 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:09.822076 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7687c3ed-944f-49f8-a21c-d971b0d11935 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:09.822280 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e668a8f8-c27e-44d4-a466-c6d497f464c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:12:09.829681 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-e668a8f8-c27e-44d4-a466-c6d497f464c3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:12:09.829681 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:09.829681 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:09.829681 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:09.829681 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:09.829681 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:09.829681 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:09.829681 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:09.829681 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:09.829681 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:09.829681 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:09.829681 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:09.829681 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:12:09.829681 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:12:09.829681 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:12:09.829681 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:12:09.830858 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:12:09.830858 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:12:09.830858 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:12:09.830858 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:12:09.830858 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:09.830858 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:09.830858 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:09.830858 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:09.830858 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:09.830858 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:09.830858 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:09.830858 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:09.830858 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:09.830858 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:09.830858 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:09.830858 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:09.831653 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:09.831653 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:09.831653 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:09.831653 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:09.831653 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:12:09.831653 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:12:09.831653 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:12:09.831653 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:12:09.831653 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:12:09.831653 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:12:09.831653 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:12:09.831653 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:12:09.832798 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 174/361] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:12:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:09.834348 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e668a8f8-c27e-44d4-a466-c6d497f464c3 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:09.860608 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:09.922788 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f70d6d-0a3c-421d-a188-14456fb5d4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:09.923939 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f70d6d-0a3c-421d-a188-14456fb5d4be None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:09.924412 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f70d6d-0a3c-421d-a188-14456fb5d4be None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:09.924854 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f70d6d-0a3c-421d-a188-14456fb5d4be None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:09.926310 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f70d6d-0a3c-421d-a188-14456fb5d4be None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:09.926853 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f70d6d-0a3c-421d-a188-14456fb5d4be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:09.929476 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f70d6d-0a3c-421d-a188-14456fb5d4be None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:09.952005 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 188/362] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:12:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:09.954061 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f70d6d-0a3c-421d-a188-14456fb5d4be None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:10.055926 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e668a8f8-c27e-44d4-a466-c6d497f464c3 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:10.130275 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f8ff22-9107-4c42-806c-cbfaf0544245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:10.131954 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f8ff22-9107-4c42-806c-cbfaf0544245 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:10.132393 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f8ff22-9107-4c42-806c-cbfaf0544245 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:10.133291 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f8ff22-9107-4c42-806c-cbfaf0544245 None admin] PATH_INFO: `/v3/groups/b95be051ba524e2986ba6efc42648e76` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:10.137412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f8ff22-9107-4c42-806c-cbfaf0544245 None admin] RBAC: Authorizing `identity:get_group(group_id=b95be051ba524e2986ba6efc42648e76)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:10.140260 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f8ff22-9107-4c42-806c-cbfaf0544245 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:10.144348 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 175/363] 10.222.0.87 () {56 vars in 1180 bytes} [Tue Jun 21 18:12:09 2022] GET /identity/v3/groups/b95be051ba524e2986ba6efc42648e76 => generated 226 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:10.146182 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f8ff22-9107-4c42-806c-cbfaf0544245 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:10.161687 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f70d6d-0a3c-421d-a188-14456fb5d4be None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:10.211817 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54531e0-b529-4e93-942a-9893e6b66668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:10.213410 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54531e0-b529-4e93-942a-9893e6b66668 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:10.213957 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54531e0-b529-4e93-942a-9893e6b66668 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:10.214454 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54531e0-b529-4e93-942a-9893e6b66668 None admin] PATH_INFO: `/v3/projects/ee325f0b1b08455686f9e0a696ca57c8` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:10.218472 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54531e0-b529-4e93-942a-9893e6b66668 None admin] RBAC: Authorizing `identity:get_project(project_id=ee325f0b1b08455686f9e0a696ca57c8)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:10.221269 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54531e0-b529-4e93-942a-9893e6b66668 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:10.224651 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 189/364] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:12:09 2022] GET /identity/v3/projects/ee325f0b1b08455686f9e0a696ca57c8 => generated 302 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:10.226143 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54531e0-b529-4e93-942a-9893e6b66668 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:10.275715 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:10.320479 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:10.361241 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f8ff22-9107-4c42-806c-cbfaf0544245 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:10.408568 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54531e0-b529-4e93-942a-9893e6b66668 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:10.430942 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50fa10-b72a-436c-94e1-b0196d3bd732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:10.431826 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50fa10-b72a-436c-94e1-b0196d3bd732 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:10.432810 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50fa10-b72a-436c-94e1-b0196d3bd732 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:10.432810 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50fa10-b72a-436c-94e1-b0196d3bd732 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:10.451614 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e50fa10-b72a-436c-94e1-b0196d3bd732 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:12:10.460994 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-2e50fa10-b72a-436c-94e1-b0196d3bd732 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:12:10.460994 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:10.460994 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:10.460994 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:10.460994 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:10.460994 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:10.460994 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:10.460994 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:10.460994 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:10.460994 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:10.460994 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:10.460994 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:10.460994 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:12:10.460994 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:12:10.460994 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:12:10.460994 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:12:10.463070 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:12:10.463070 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:12:10.463070 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:12:10.463070 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:12:10.463070 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:10.463070 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:10.463070 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:10.463070 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:10.463070 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:10.463070 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:10.463070 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:10.463070 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:10.463070 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:10.463070 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:10.463070 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:10.463070 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:10.464721 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:10.464721 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:10.464721 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:10.464721 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:10.464721 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:12:10.464721 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:12:10.464721 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:12:10.464721 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:12:10.464721 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:12:10.464721 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:12:10.464721 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:12:10.464721 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:12:10.464721 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 176/365] 10.222.0.87 () {56 vars in 1096 bytes} [Tue Jun 21 18:12:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:10.464721 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50fa10-b72a-436c-94e1-b0196d3bd732 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:10.493370 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb3552-d5e8-4449-a708-eee2649bc12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:10.494501 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb3552-d5e8-4449-a708-eee2649bc12d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:10.494929 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb3552-d5e8-4449-a708-eee2649bc12d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:10.495349 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb3552-d5e8-4449-a708-eee2649bc12d None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:10.496623 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddb3552-d5e8-4449-a708-eee2649bc12d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:10.497092 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddb3552-d5e8-4449-a708-eee2649bc12d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:10.499507 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddb3552-d5e8-4449-a708-eee2649bc12d None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:10.522557 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 190/366] 10.222.0.87 () {56 vars in 1101 bytes} [Tue Jun 21 18:12:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:10.522557 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb3552-d5e8-4449-a708-eee2649bc12d None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:10.662183 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50fa10-b72a-436c-94e1-b0196d3bd732 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:10.685360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:10.705354 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb3552-d5e8-4449-a708-eee2649bc12d None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:10.737308 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95c2d2a-5438-4065-b2b0-dadadcf9177e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:10.738486 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95c2d2a-5438-4065-b2b0-dadadcf9177e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:10.738969 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95c2d2a-5438-4065-b2b0-dadadcf9177e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:10.739470 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95c2d2a-5438-4065-b2b0-dadadcf9177e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:10.762878 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:10.770182 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41edfc60-68b9-451c-99d4-870580d86170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:10.770932 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edfc60-68b9-451c-99d4-870580d86170 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:10.771263 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edfc60-68b9-451c-99d4-870580d86170 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:10.771533 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edfc60-68b9-451c-99d4-870580d86170 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:10.772498 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41edfc60-68b9-451c-99d4-870580d86170 None admin] RBAC: Adding query filter params (group.id=b95be051ba524e2986ba6efc42648e76, role.id=46e77bb17fc3465496a7a3a29d210042, scope.project.id=48caa0e9d7fe40d28e99abd1a7a6896f) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:10.772714 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41edfc60-68b9-451c-99d4-870580d86170 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:10.774343 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41edfc60-68b9-451c-99d4-870580d86170 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:10.793679 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 191/367] 10.222.0.87 () {56 vars in 1376 bytes} [Tue Jun 21 18:12:10 2022] GET /identity/v3/role_assignments?group.id=b95be051ba524e2986ba6efc42648e76&scope.project.id=48caa0e9d7fe40d28e99abd1a7a6896f&role.id=46e77bb17fc3465496a7a3a29d210042 => generated 260 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:10.793679 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41edfc60-68b9-451c-99d4-870580d86170 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:10.918941 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95c2d2a-5438-4065-b2b0-dadadcf9177e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:10.922434 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95c2d2a-5438-4065-b2b0-dadadcf9177e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:10.974347 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41edfc60-68b9-451c-99d4-870580d86170 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:11.046556 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0ef31-06f9-4d16-ac9e-c145c472c58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:11.049069 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0ef31-06f9-4d16-ac9e-c145c472c58d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:11.049582 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0ef31-06f9-4d16-ac9e-c145c472c58d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:11.050145 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0ef31-06f9-4d16-ac9e-c145c472c58d None admin] PATH_INFO: `/v3/groups/b95be051ba524e2986ba6efc42648e76` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:11.052706 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb0ef31-06f9-4d16-ac9e-c145c472c58d None admin] RBAC: Authorizing `identity:get_group(group_id=b95be051ba524e2986ba6efc42648e76)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:11.055888 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb0ef31-06f9-4d16-ac9e-c145c472c58d None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:11.066565 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 192/368] 10.222.0.87 () {56 vars in 1180 bytes} [Tue Jun 21 18:12:10 2022] GET /identity/v3/groups/b95be051ba524e2986ba6efc42648e76 => generated 226 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:11.068469 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0ef31-06f9-4d16-ac9e-c145c472c58d None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:11.121510 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:11.131048 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 177/369] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:12:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 668 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:12:11.132696 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95c2d2a-5438-4065-b2b0-dadadcf9177e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:11.234312 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:11.311546 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0ef31-06f9-4d16-ac9e-c145c472c58d None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:11.377972 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95c2d2a-5438-4065-b2b0-dadadcf9177e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:11.392944 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343ad93c-73d2-411a-9855-a7d4ddd368b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:11.394772 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ad93c-73d2-411a-9855-a7d4ddd368b6 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:11.395445 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ad93c-73d2-411a-9855-a7d4ddd368b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:11.395949 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ad93c-73d2-411a-9855-a7d4ddd368b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:11.397367 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343ad93c-73d2-411a-9855-a7d4ddd368b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:11.398141 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343ad93c-73d2-411a-9855-a7d4ddd368b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:11.400696 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343ad93c-73d2-411a-9855-a7d4ddd368b6 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:11.417577 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 193/370] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:12:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:11.418995 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343ad93c-73d2-411a-9855-a7d4ddd368b6 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:11.450704 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18620b04-75cc-40a2-ad45-3503c94e9494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:11.452295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18620b04-75cc-40a2-ad45-3503c94e9494 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:11.452800 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18620b04-75cc-40a2-ad45-3503c94e9494 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:11.453290 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18620b04-75cc-40a2-ad45-3503c94e9494 None admin] PATH_INFO: `/v3/users/a5f2f921ad744fbbb3c102f92dda5360` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:11.456777 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18620b04-75cc-40a2-ad45-3503c94e9494 None admin] RBAC: Authorizing `identity:get_user(user_id=a5f2f921ad744fbbb3c102f92dda5360)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:11.459591 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18620b04-75cc-40a2-ad45-3503c94e9494 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:11.462764 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 178/371] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:12:11 2022] GET /identity/v3/users/a5f2f921ad744fbbb3c102f92dda5360 => generated 293 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:11.465711 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18620b04-75cc-40a2-ad45-3503c94e9494 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:11.556223 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:11.579631 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343ad93c-73d2-411a-9855-a7d4ddd368b6 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:11.626508 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe689dc-7c4e-4733-8049-dccbfb1e9658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:11.627490 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe689dc-7c4e-4733-8049-dccbfb1e9658 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:11.627897 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe689dc-7c4e-4733-8049-dccbfb1e9658 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:11.628350 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe689dc-7c4e-4733-8049-dccbfb1e9658 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:11.629702 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe689dc-7c4e-4733-8049-dccbfb1e9658 None admin] RBAC: Adding query filter params (group.id=b95be051ba524e2986ba6efc42648e76, role.id=da6d4a0ab8c04809ba120170fb4ae8a9, scope.project.id=ee325f0b1b08455686f9e0a696ca57c8) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:11.630152 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe689dc-7c4e-4733-8049-dccbfb1e9658 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:11.631869 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe689dc-7c4e-4733-8049-dccbfb1e9658 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:11.649678 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 194/372] 10.222.0.87 () {56 vars in 1376 bytes} [Tue Jun 21 18:12:11 2022] GET /identity/v3/role_assignments?group.id=b95be051ba524e2986ba6efc42648e76&scope.project.id=ee325f0b1b08455686f9e0a696ca57c8&role.id=da6d4a0ab8c04809ba120170fb4ae8a9 => generated 260 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:11.649678 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7bf3a-45be-4d7d-8dcc-1a9dcabf2865 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:11.649678 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7bf3a-45be-4d7d-8dcc-1a9dcabf2865 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:11.649678 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7bf3a-45be-4d7d-8dcc-1a9dcabf2865 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:11.649678 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 195/373] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:12:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:12:11.649678 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d7bf3a-45be-4d7d-8dcc-1a9dcabf2865 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:11.670454 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18620b04-75cc-40a2-ad45-3503c94e9494 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:11.689052 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:11.737820 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e5670-960c-4d5d-b6a5-c1ec144bb86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:11.739574 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e5670-960c-4d5d-b6a5-c1ec144bb86e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:11.740130 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e5670-960c-4d5d-b6a5-c1ec144bb86e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:11.740674 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e5670-960c-4d5d-b6a5-c1ec144bb86e None admin] PATH_INFO: `/v3/projects/ee325f0b1b08455686f9e0a696ca57c8` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:11.742991 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12e5670-960c-4d5d-b6a5-c1ec144bb86e None admin] RBAC: Authorizing `identity:get_project(project_id=ee325f0b1b08455686f9e0a696ca57c8)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:11.745919 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12e5670-960c-4d5d-b6a5-c1ec144bb86e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:11.749256 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 179/374] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:12:11 2022] GET /identity/v3/projects/ee325f0b1b08455686f9e0a696ca57c8 => generated 302 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:11.751253 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e5670-960c-4d5d-b6a5-c1ec144bb86e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:11.874453 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d7bf3a-45be-4d7d-8dcc-1a9dcabf2865 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:11.960331 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e7580-5e3d-4579-b9a6-42b10e356ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:11.961576 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e7580-5e3d-4579-b9a6-42b10e356ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:11.962098 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e7580-5e3d-4579-b9a6-42b10e356ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:11.962621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e7580-5e3d-4579-b9a6-42b10e356ee4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:11.970930 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e5670-960c-4d5d-b6a5-c1ec144bb86e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:12.003445 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:12.046007 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d133c1-292a-4552-a36a-0bd564b69367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:12.047675 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d133c1-292a-4552-a36a-0bd564b69367 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:12.048371 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d133c1-292a-4552-a36a-0bd564b69367 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:12.049130 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d133c1-292a-4552-a36a-0bd564b69367 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:12.051293 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d133c1-292a-4552-a36a-0bd564b69367 None admin] RBAC: Adding query filter params (role.id=5efb4931668c4edf87773a1f5d47f005, user.id=a5f2f921ad744fbbb3c102f92dda5360, scope.system=all) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:12.052165 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d133c1-292a-4552-a36a-0bd564b69367 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:12.055447 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d133c1-292a-4552-a36a-0bd564b69367 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:12.090508 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 180/375] 10.222.0.87 () {56 vars in 1308 bytes} [Tue Jun 21 18:12:11 2022] GET /identity/v3/role_assignments?user.id=a5f2f921ad744fbbb3c102f92dda5360&scope.system=all&role.id=5efb4931668c4edf87773a1f5d47f005 => generated 510 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:12.093568 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93308c98-5fbe-462c-889f-c7ca0c13d1fd None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:12.094449 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93308c98-5fbe-462c-889f-c7ca0c13d1fd None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:12.095252 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93308c98-5fbe-462c-889f-c7ca0c13d1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:12.115308 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941e7580-5e3d-4579-b9a6-42b10e356ee4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:12.117123 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941e7580-5e3d-4579-b9a6-42b10e356ee4 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:12.160048 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-93308c98-5fbe-462c-889f-c7ca0c13d1fd None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:12:12.163505 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-93308c98-5fbe-462c-889f-c7ca0c13d1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:12:12.278414 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:12.409932 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 181/376] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:12:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:12:12.411533 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93308c98-5fbe-462c-889f-c7ca0c13d1fd None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:12.433320 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 196/377] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:12:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 786 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:12:12.443578 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e7580-5e3d-4579-b9a6-42b10e356ee4 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:12.551061 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:12.657780 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e7580-5e3d-4579-b9a6-42b10e356ee4 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:12.668790 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93308c98-5fbe-462c-889f-c7ca0c13d1fd None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:12.729666 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35d5946-1a9f-42e5-a791-803f49e1fc0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:12.732085 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d5946-1a9f-42e5-a791-803f49e1fc0c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:12.732770 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d5946-1a9f-42e5-a791-803f49e1fc0c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:12.733456 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d5946-1a9f-42e5-a791-803f49e1fc0c None admin] PATH_INFO: `/v3/projects/58da051d3b4a47fab4c706fb517e9ae8` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:12.736830 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35d5946-1a9f-42e5-a791-803f49e1fc0c None admin] RBAC: Authorizing `identity:get_project(project_id=58da051d3b4a47fab4c706fb517e9ae8)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:12.739970 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35d5946-1a9f-42e5-a791-803f49e1fc0c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:12.744398 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 197/378] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:12:12 2022] GET /identity/v3/projects/58da051d3b4a47fab4c706fb517e9ae8 => generated 344 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:12.746270 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35d5946-1a9f-42e5-a791-803f49e1fc0c None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:12.751698 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b50e92-eb00-4707-b338-6b60b145419b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:12.753436 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b50e92-eb00-4707-b338-6b60b145419b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:12.755201 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b50e92-eb00-4707-b338-6b60b145419b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:12.755712 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b50e92-eb00-4707-b338-6b60b145419b None admin] PATH_INFO: `/v3/groups/b95be051ba524e2986ba6efc42648e76` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:12.758111 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b50e92-eb00-4707-b338-6b60b145419b None admin] RBAC: Authorizing `identity:get_group(group_id=b95be051ba524e2986ba6efc42648e76)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:12.761185 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b50e92-eb00-4707-b338-6b60b145419b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:12.769854 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 182/379] 10.222.0.87 () {56 vars in 1180 bytes} [Tue Jun 21 18:12:12 2022] GET /identity/v3/groups/b95be051ba524e2986ba6efc42648e76 => generated 226 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:12.771677 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b50e92-eb00-4707-b338-6b60b145419b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:12.851219 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:12.979095 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35d5946-1a9f-42e5-a791-803f49e1fc0c None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:13.017296 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:13.023573 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b50e92-eb00-4707-b338-6b60b145419b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:13.068086 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcbcdec-789a-4845-8f18-a9ebf1cb5bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:13.070178 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbcdec-789a-4845-8f18-a9ebf1cb5bc4 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:13.071006 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbcdec-789a-4845-8f18-a9ebf1cb5bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:13.071765 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbcdec-789a-4845-8f18-a9ebf1cb5bc4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:13.110629 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcbcdec-789a-4845-8f18-a9ebf1cb5bc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:12:13.123055 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d22de6-a724-4775-91f4-6524e66a8442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:13.124779 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d22de6-a724-4775-91f4-6524e66a8442 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:13.125324 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d22de6-a724-4775-91f4-6524e66a8442 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:13.126789 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d22de6-a724-4775-91f4-6524e66a8442 None admin] PATH_INFO: `/v3/groups/d31a036caa574bbc943d9eabac35f5b2` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:13.130811 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-4bcbcdec-789a-4845-8f18-a9ebf1cb5bc4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:12:13.130811 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:13.130811 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:13.130811 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:13.130811 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:13.130811 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:13.130811 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:13.130811 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:13.130811 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:13.130811 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:13.130811 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:13.130811 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:13.130811 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:12:13.130811 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:12:13.130811 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:12:13.130811 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:12:13.132614 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:12:13.132614 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:12:13.132614 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:12:13.132614 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:12:13.132614 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:13.132614 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:13.132614 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:13.132614 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:13.132614 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:13.132614 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:13.132614 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:13.132614 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:13.132614 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:13.132614 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:13.132614 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:13.132614 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:13.134254 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:13.134254 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:13.134254 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:13.134254 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:13.134254 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:12:13.134254 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:12:13.134254 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:12:13.134254 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:12:13.134254 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:12:13.134254 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:12:13.134254 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:12:13.134254 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:12:13.138461 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 198/380] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:12:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 393 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:13.140389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcbcdec-789a-4845-8f18-a9ebf1cb5bc4 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:13.149045 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d22de6-a724-4775-91f4-6524e66a8442 None admin] RBAC: Authorizing `identity:get_group(group_id=d31a036caa574bbc943d9eabac35f5b2)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:13.152038 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d22de6-a724-4775-91f4-6524e66a8442 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:13.160472 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 183/381] 10.222.0.87 () {56 vars in 1180 bytes} [Tue Jun 21 18:12:12 2022] GET /identity/v3/groups/d31a036caa574bbc943d9eabac35f5b2 => generated 229 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:13.162889 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d22de6-a724-4775-91f4-6524e66a8442 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:13.378075 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcbcdec-789a-4845-8f18-a9ebf1cb5bc4 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:13.418134 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:13.431027 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d22de6-a724-4775-91f4-6524e66a8442 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:13.449922 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa53850-cbee-48b7-91e7-5cd2a404e573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:13.450710 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa53850-cbee-48b7-91e7-5cd2a404e573 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:13.451006 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa53850-cbee-48b7-91e7-5cd2a404e573 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:13.451292 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa53850-cbee-48b7-91e7-5cd2a404e573 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:13.452200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa53850-cbee-48b7-91e7-5cd2a404e573 None admin] RBAC: Adding query filter params (group.id=b95be051ba524e2986ba6efc42648e76, role.id=46e77bb17fc3465496a7a3a29d210042, scope.project.id=ee325f0b1b08455686f9e0a696ca57c8) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:13.452524 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa53850-cbee-48b7-91e7-5cd2a404e573 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:13.455463 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa53850-cbee-48b7-91e7-5cd2a404e573 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:13.464184 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 199/382] 10.222.0.87 () {56 vars in 1376 bytes} [Tue Jun 21 18:12:13 2022] GET /identity/v3/role_assignments?group.id=b95be051ba524e2986ba6efc42648e76&scope.project.id=ee325f0b1b08455686f9e0a696ca57c8&role.id=46e77bb17fc3465496a7a3a29d210042 => generated 260 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:13.465272 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa53850-cbee-48b7-91e7-5cd2a404e573 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:13.527765 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d36e14-23a7-446e-adb9-e917c4f26a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:13.528808 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d36e14-23a7-446e-adb9-e917c4f26a15 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:13.529771 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d36e14-23a7-446e-adb9-e917c4f26a15 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:13.530192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d36e14-23a7-446e-adb9-e917c4f26a15 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:13.531309 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d36e14-23a7-446e-adb9-e917c4f26a15 None admin] RBAC: Adding query filter params (name=member) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:13.532013 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d36e14-23a7-446e-adb9-e917c4f26a15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:13.535085 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d36e14-23a7-446e-adb9-e917c4f26a15 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:13.566432 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 184/383] 10.222.0.87 () {56 vars in 1101 bytes} [Tue Jun 21 18:12:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 405 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:13.575226 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d36e14-23a7-446e-adb9-e917c4f26a15 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:13.603907 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:13.798998 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa53850-cbee-48b7-91e7-5cd2a404e573 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:13.908965 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d36e14-23a7-446e-adb9-e917c4f26a15 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:13.959181 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39914c8-c29a-47c8-a4e4-b9da7a3e6100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:13.960510 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39914c8-c29a-47c8-a4e4-b9da7a3e6100 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:13.961041 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39914c8-c29a-47c8-a4e4-b9da7a3e6100 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:13.961369 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39914c8-c29a-47c8-a4e4-b9da7a3e6100 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:13.962834 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39914c8-c29a-47c8-a4e4-b9da7a3e6100 None admin] RBAC: Adding query filter params (group.id=d31a036caa574bbc943d9eabac35f5b2, role.id=98f0943a610f4a9e9f8de54f6e9f717d, scope.project.id=58da051d3b4a47fab4c706fb517e9ae8) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:13.963168 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39914c8-c29a-47c8-a4e4-b9da7a3e6100 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:13.965995 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39914c8-c29a-47c8-a4e4-b9da7a3e6100 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:13.991341 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 200/384] 10.222.0.87 () {56 vars in 1376 bytes} [Tue Jun 21 18:12:13 2022] GET /identity/v3/role_assignments?group.id=d31a036caa574bbc943d9eabac35f5b2&scope.project.id=58da051d3b4a47fab4c706fb517e9ae8&role.id=98f0943a610f4a9e9f8de54f6e9f717d => generated 260 bytes in 526 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:13.994613 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc545055-ece0-4f69-aa5c-be6f2440df81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:13.996188 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc545055-ece0-4f69-aa5c-be6f2440df81 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:13.996638 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc545055-ece0-4f69-aa5c-be6f2440df81 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:13.997082 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc545055-ece0-4f69-aa5c-be6f2440df81 None admin] PATH_INFO: `/v3/groups/b95be051ba524e2986ba6efc42648e76` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:13.999393 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc545055-ece0-4f69-aa5c-be6f2440df81 None admin] RBAC: Authorizing `identity:get_group(group_id=b95be051ba524e2986ba6efc42648e76)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:14.002388 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc545055-ece0-4f69-aa5c-be6f2440df81 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:14.006223 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 185/385] 10.222.0.87 () {56 vars in 1180 bytes} [Tue Jun 21 18:12:13 2022] GET /identity/v3/groups/b95be051ba524e2986ba6efc42648e76 => generated 226 bytes in 432 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:14.014374 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39914c8-c29a-47c8-a4e4-b9da7a3e6100 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:14.283388 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39914c8-c29a-47c8-a4e4-b9da7a3e6100 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:14.301162 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:14.356106 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e410de-985b-4d9f-b417-67f4da19f7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:14.357717 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e410de-985b-4d9f-b417-67f4da19f7c6 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:14.358418 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e410de-985b-4d9f-b417-67f4da19f7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:14.358921 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e410de-985b-4d9f-b417-67f4da19f7c6 None admin] PATH_INFO: `/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:14.361265 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e410de-985b-4d9f-b417-67f4da19f7c6 None admin] RBAC: Authorizing `identity:get_project(project_id=48caa0e9d7fe40d28e99abd1a7a6896f)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:14.364179 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e410de-985b-4d9f-b417-67f4da19f7c6 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:14.373895 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 201/386] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:12:14 2022] GET /identity/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f => generated 298 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:14.379487 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc545055-ece0-4f69-aa5c-be6f2440df81 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:14.437251 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241d05e-11a1-4b4f-803f-2c6b7ca44562 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:14.438286 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241d05e-11a1-4b4f-803f-2c6b7ca44562 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:14.438779 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241d05e-11a1-4b4f-803f-2c6b7ca44562 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:14.441039 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 202/387] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:12:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:12:14.449627 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2395da5-68d1-4ed1-9a04-1d82f285122b None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:14.450539 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2395da5-68d1-4ed1-9a04-1d82f285122b None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:14.451067 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2395da5-68d1-4ed1-9a04-1d82f285122b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:14.467782 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:14.486634 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-c2395da5-68d1-4ed1-9a04-1d82f285122b None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:12:14.488484 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c2395da5-68d1-4ed1-9a04-1d82f285122b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:12:14.813097 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 203/388] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:12:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:12:14.889137 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2395da5-68d1-4ed1-9a04-1d82f285122b None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:14.914206 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc545055-ece0-4f69-aa5c-be6f2440df81 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:15.107917 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f53074-4e85-4d17-9322-b1a66012f327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:15.109517 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f53074-4e85-4d17-9322-b1a66012f327 None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:15.111935 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f53074-4e85-4d17-9322-b1a66012f327 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:15.111935 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f53074-4e85-4d17-9322-b1a66012f327 None admin] PATH_INFO: `/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f/groups/b95be051ba524e2986ba6efc42648e76/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:15.129946 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f53074-4e85-4d17-9322-b1a66012f327 None admin] RBAC: Authorizing `identity:create_grant(project_id=48caa0e9d7fe40d28e99abd1a7a6896f, group_id=b95be051ba524e2986ba6efc42648e76, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:15.132691 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f53074-4e85-4d17-9322-b1a66012f327 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:15.199102 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 186/389] 10.222.0.87 () {58 vars in 1442 bytes} [Tue Jun 21 18:12:14 2022] PUT /identity/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f/groups/b95be051ba524e2986ba6efc42648e76/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 820 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:12:15.201420 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a0f73-ade7-4d79-8238-3c162bc1f685 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:15.201907 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a0f73-ade7-4d79-8238-3c162bc1f685 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:15.202343 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a0f73-ade7-4d79-8238-3c162bc1f685 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:15.205613 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 187/390] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:12:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:12:15.221837 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca671d9-4a62-4389-b6b7-7b33053805c0 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:15.222317 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca671d9-4a62-4389-b6b7-7b33053805c0 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:15.222737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca671d9-4a62-4389-b6b7-7b33053805c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:15.228846 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2395da5-68d1-4ed1-9a04-1d82f285122b None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:15.229866 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:15.290658 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-bca671d9-4a62-4389-b6b7-7b33053805c0 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:12:15.298307 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-bca671d9-4a62-4389-b6b7-7b33053805c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:12:15.305311 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e75dbd4-9e81-47a6-bea6-c683cc32d8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:15.307369 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75dbd4-9e81-47a6-bea6-c683cc32d8cb None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:15.309682 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75dbd4-9e81-47a6-bea6-c683cc32d8cb None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:15.310551 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75dbd4-9e81-47a6-bea6-c683cc32d8cb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:15.352130 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e75dbd4-9e81-47a6-bea6-c683cc32d8cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:12:15.384906 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-0e75dbd4-9e81-47a6-bea6-c683cc32d8cb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:12:15.384906 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:15.384906 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:15.384906 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:15.384906 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:15.384906 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:15.384906 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:15.384906 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:15.384906 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:15.384906 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:15.384906 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:15.384906 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:15.384906 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:12:15.384906 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:12:15.384906 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:12:15.384906 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:12:15.386383 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:12:15.386383 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:12:15.386383 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:12:15.386383 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:12:15.386383 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:15.386383 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:15.386383 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:15.386383 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:15.386383 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:15.386383 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:15.386383 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:15.386383 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:15.386383 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:15.386383 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:15.386383 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:15.386383 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:15.387434 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:15.387434 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:15.387434 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:15.387434 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:15.387434 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:12:15.387434 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:12:15.387434 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:12:15.387434 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:12:15.387434 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:12:15.387434 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:12:15.387434 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:12:15.387434 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:12:15.393546 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 204/391] 10.222.0.87 () {56 vars in 1114 bytes} [Tue Jun 21 18:12:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 507 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:12:15.403485 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e75dbd4-9e81-47a6-bea6-c683cc32d8cb None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:15.552444 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 188/392] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:12:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:12:15.596719 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca671d9-4a62-4389-b6b7-7b33053805c0 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:15.660113 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:15.669481 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e75dbd4-9e81-47a6-bea6-c683cc32d8cb None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:15.796772 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7a110d-c591-43df-84ed-cc8aaf320246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:15.798983 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a110d-c591-43df-84ed-cc8aaf320246 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:15.799451 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a110d-c591-43df-84ed-cc8aaf320246 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:15.799926 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a110d-c591-43df-84ed-cc8aaf320246 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:15.804142 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7a110d-c591-43df-84ed-cc8aaf320246 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:15.804681 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7a110d-c591-43df-84ed-cc8aaf320246 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:15.807439 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7a110d-c591-43df-84ed-cc8aaf320246 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:15.847704 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 205/393] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:12:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 445 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:15.861575 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7a110d-c591-43df-84ed-cc8aaf320246 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:15.970919 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:16.003067 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca671d9-4a62-4389-b6b7-7b33053805c0 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:16.176864 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7a110d-c591-43df-84ed-cc8aaf320246 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:16.192062 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a5a359-5517-4892-b14b-3e4def02dc15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:16.193499 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5a359-5517-4892-b14b-3e4def02dc15 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:16.194039 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5a359-5517-4892-b14b-3e4def02dc15 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:16.194531 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5a359-5517-4892-b14b-3e4def02dc15 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:16.216716 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a5a359-5517-4892-b14b-3e4def02dc15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:12:16.233816 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-41a5a359-5517-4892-b14b-3e4def02dc15 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:12:16.233816 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:16.233816 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:16.233816 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:16.233816 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:16.233816 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:16.233816 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:16.233816 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:16.233816 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:16.233816 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:16.233816 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:16.233816 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:16.233816 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:12:16.233816 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:12:16.233816 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:12:16.233816 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:12:16.235957 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:12:16.235957 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:12:16.235957 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:12:16.235957 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:12:16.235957 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:16.235957 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:16.235957 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:16.235957 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:16.235957 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:16.235957 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:16.235957 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:16.235957 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:16.235957 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:16.235957 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:16.235957 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:16.235957 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:16.237452 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:16.237452 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:16.237452 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:16.237452 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:16.237452 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:12:16.237452 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:12:16.237452 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:12:16.237452 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:12:16.237452 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:12:16.237452 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:12:16.237452 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:12:16.237452 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:12:16.237452 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 189/394] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:12:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 641 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:16.246798 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a5a359-5517-4892-b14b-3e4def02dc15 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:16.301477 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabe3e76-e66f-4bb2-96e4-215ed77ebfe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:16.303082 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabe3e76-e66f-4bb2-96e4-215ed77ebfe4 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:16.303892 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabe3e76-e66f-4bb2-96e4-215ed77ebfe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:16.304432 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabe3e76-e66f-4bb2-96e4-215ed77ebfe4 None admin] PATH_INFO: `/v3/groups/b95be051ba524e2986ba6efc42648e76` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:16.310380 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabe3e76-e66f-4bb2-96e4-215ed77ebfe4 None admin] RBAC: Authorizing `identity:get_group(group_id=b95be051ba524e2986ba6efc42648e76)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:16.313294 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabe3e76-e66f-4bb2-96e4-215ed77ebfe4 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:16.316952 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 206/395] 10.222.0.87 () {56 vars in 1180 bytes} [Tue Jun 21 18:12:15 2022] GET /identity/v3/groups/b95be051ba524e2986ba6efc42648e76 => generated 226 bytes in 458 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:12:16.331113 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabe3e76-e66f-4bb2-96e4-215ed77ebfe4 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:16.457572 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:16.512673 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a5a359-5517-4892-b14b-3e4def02dc15 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:16.593721 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:16.599604 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b703b9ba-3458-4bae-9cc2-c768881fce30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:16.600730 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703b9ba-3458-4bae-9cc2-c768881fce30 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:16.601211 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703b9ba-3458-4bae-9cc2-c768881fce30 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:16.601717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703b9ba-3458-4bae-9cc2-c768881fce30 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:16.603040 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b703b9ba-3458-4bae-9cc2-c768881fce30 None admin] RBAC: Adding query filter params (name=member) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:16.603563 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b703b9ba-3458-4bae-9cc2-c768881fce30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:16.606573 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b703b9ba-3458-4bae-9cc2-c768881fce30 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:16.623695 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 190/396] 10.222.0.87 () {56 vars in 1101 bytes} [Tue Jun 21 18:12:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:12:16.627103 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabe3e76-e66f-4bb2-96e4-215ed77ebfe4 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:16.637707 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b703b9ba-3458-4bae-9cc2-c768881fce30 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:16.738964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe51368-8d58-46b9-88e4-e10a458e3cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:16.740196 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe51368-8d58-46b9-88e4-e10a458e3cb8 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:16.740480 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe51368-8d58-46b9-88e4-e10a458e3cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:16.740760 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe51368-8d58-46b9-88e4-e10a458e3cb8 None admin] PATH_INFO: `/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:16.742193 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe51368-8d58-46b9-88e4-e10a458e3cb8 None admin] RBAC: Authorizing `identity:get_project(project_id=48caa0e9d7fe40d28e99abd1a7a6896f)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:16.743705 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe51368-8d58-46b9-88e4-e10a458e3cb8 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:16.746172 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 207/397] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:12:16 2022] GET /identity/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f => generated 298 bytes in 416 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:16.754796 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe51368-8d58-46b9-88e4-e10a458e3cb8 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:16.861998 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b703b9ba-3458-4bae-9cc2-c768881fce30 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:16.934175 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f50ef-b481-4c3a-99a6-494d2a4ed62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:16.935717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f50ef-b481-4c3a-99a6-494d2a4ed62c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:16.936248 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f50ef-b481-4c3a-99a6-494d2a4ed62c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:16.936726 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f50ef-b481-4c3a-99a6-494d2a4ed62c None admin] PATH_INFO: `/v3/groups/b95be051ba524e2986ba6efc42648e76` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:16.939862 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51f50ef-b481-4c3a-99a6-494d2a4ed62c None admin] RBAC: Authorizing `identity:get_group(group_id=b95be051ba524e2986ba6efc42648e76)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:16.942782 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51f50ef-b481-4c3a-99a6-494d2a4ed62c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:16.946580 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 191/398] 10.222.0.87 () {56 vars in 1180 bytes} [Tue Jun 21 18:12:16 2022] GET /identity/v3/groups/b95be051ba524e2986ba6efc42648e76 => generated 226 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:16.956595 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f50ef-b481-4c3a-99a6-494d2a4ed62c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:16.975354 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:17.054396 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe51368-8d58-46b9-88e4-e10a458e3cb8 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:17.155067 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331c6ea-4514-4ac6-a005-7aea2dd4557f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:17.156561 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331c6ea-4514-4ac6-a005-7aea2dd4557f None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:17.157052 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331c6ea-4514-4ac6-a005-7aea2dd4557f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:17.157552 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331c6ea-4514-4ac6-a005-7aea2dd4557f None admin] PATH_INFO: `/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f/groups/b95be051ba524e2986ba6efc42648e76/roles/46e77bb17fc3465496a7a3a29d210042` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:17.165221 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e331c6ea-4514-4ac6-a005-7aea2dd4557f None admin] RBAC: Authorizing `identity:create_grant(project_id=48caa0e9d7fe40d28e99abd1a7a6896f, group_id=b95be051ba524e2986ba6efc42648e76, role_id=46e77bb17fc3465496a7a3a29d210042)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:17.167223 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e331c6ea-4514-4ac6-a005-7aea2dd4557f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:17.192523 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f50ef-b481-4c3a-99a6-494d2a4ed62c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:17.201579 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 208/399] 10.222.0.87 () {58 vars in 1442 bytes} [Tue Jun 21 18:12:16 2022] PUT /identity/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f/groups/b95be051ba524e2986ba6efc42648e76/roles/46e77bb17fc3465496a7a3a29d210042 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:12:17.268407 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b665d53c-32ca-4280-89e4-cc287cae0c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:17.270526 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665d53c-32ca-4280-89e4-cc287cae0c72 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:17.271346 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665d53c-32ca-4280-89e4-cc287cae0c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:17.272090 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665d53c-32ca-4280-89e4-cc287cae0c72 None admin] PATH_INFO: `/v3/projects/ee325f0b1b08455686f9e0a696ca57c8` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:17.274780 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b665d53c-32ca-4280-89e4-cc287cae0c72 None admin] RBAC: Authorizing `identity:get_project(project_id=ee325f0b1b08455686f9e0a696ca57c8)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:17.278074 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b665d53c-32ca-4280-89e4-cc287cae0c72 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:17.282503 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 192/400] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:12:16 2022] GET /identity/v3/projects/ee325f0b1b08455686f9e0a696ca57c8 => generated 302 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:12:17.292298 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331c6ea-4514-4ac6-a005-7aea2dd4557f None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:17.338359 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:17.556453 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331c6ea-4514-4ac6-a005-7aea2dd4557f None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:17.647553 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acfe9b9-6667-4c8e-9a65-773d0855ef46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:17.649189 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acfe9b9-6667-4c8e-9a65-773d0855ef46 None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:17.649739 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acfe9b9-6667-4c8e-9a65-773d0855ef46 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:17.650244 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acfe9b9-6667-4c8e-9a65-773d0855ef46 None admin] PATH_INFO: `/v3/projects/ee325f0b1b08455686f9e0a696ca57c8/groups/b95be051ba524e2986ba6efc42648e76/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:17.654150 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acfe9b9-6667-4c8e-9a65-773d0855ef46 None admin] RBAC: Authorizing `identity:create_grant(project_id=ee325f0b1b08455686f9e0a696ca57c8, group_id=b95be051ba524e2986ba6efc42648e76, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:17.657872 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acfe9b9-6667-4c8e-9a65-773d0855ef46 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:17.687174 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 209/401] 10.222.0.87 () {58 vars in 1442 bytes} [Tue Jun 21 18:12:17 2022] PUT /identity/v3/projects/ee325f0b1b08455686f9e0a696ca57c8/groups/b95be051ba524e2986ba6efc42648e76/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:12:17.722362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff186f7-de33-4a02-a79b-51aaace8a2dc None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:17.723150 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff186f7-de33-4a02-a79b-51aaace8a2dc None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:17.723498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff186f7-de33-4a02-a79b-51aaace8a2dc None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:17.726252 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 193/402] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:12:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:12:17.737255 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e1215e-8424-4b85-a192-08bf2733b01f None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:17.737799 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e1215e-8424-4b85-a192-08bf2733b01f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:17.738309 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e1215e-8424-4b85-a192-08bf2733b01f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:17.773369 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-78e1215e-8424-4b85-a192-08bf2733b01f None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:12:17.775560 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-78e1215e-8424-4b85-a192-08bf2733b01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:12:17.946437 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 210/403] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:12:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:12:17.990192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff186f7-de33-4a02-a79b-51aaace8a2dc None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:18.156710 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:18.213901 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff186f7-de33-4a02-a79b-51aaace8a2dc None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:18.286579 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7706fe-9716-49a2-a7f8-82d567a48b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:18.288137 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7706fe-9716-49a2-a7f8-82d567a48b44 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:18.288701 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7706fe-9716-49a2-a7f8-82d567a48b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:18.289209 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7706fe-9716-49a2-a7f8-82d567a48b44 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:18.311096 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7706fe-9716-49a2-a7f8-82d567a48b44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:12:18.328704 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-2f7706fe-9716-49a2-a7f8-82d567a48b44 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:12:18.328704 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:18.328704 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:18.328704 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:18.328704 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:18.328704 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:18.328704 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:18.328704 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:18.328704 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:18.328704 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:18.328704 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:18.328704 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:18.328704 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:12:18.328704 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:12:18.328704 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:12:18.328704 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:12:18.329839 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:12:18.329839 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:12:18.329839 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:12:18.329839 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:12:18.329839 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:18.329839 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:18.329839 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:18.329839 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:18.329839 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:18.329839 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:18.329839 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:18.329839 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:18.329839 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:18.329839 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:18.329839 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:18.329839 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:18.330644 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:18.330644 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:18.330644 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:18.330644 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:18.330644 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:12:18.330644 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:12:18.330644 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:12:18.330644 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:12:18.330644 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:12:18.330644 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:12:18.330644 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:12:18.330644 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:12:18.331843 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 194/404] 10.222.0.87 () {56 vars in 1114 bytes} [Tue Jun 21 18:12:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:18.341280 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e1215e-8424-4b85-a192-08bf2733b01f None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:18.369419 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:18.430189 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1971e-2a0d-4021-80a3-cd15a1e58a39 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:18.430821 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1971e-2a0d-4021-80a3-cd15a1e58a39 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:18.431301 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1971e-2a0d-4021-80a3-cd15a1e58a39 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:18.433037 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 195/405] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:12:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:12:18.440793 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9158f-5105-4b4e-a46a-7db65fb7c21f None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:18.441780 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9158f-5105-4b4e-a46a-7db65fb7c21f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:18.442468 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9158f-5105-4b4e-a46a-7db65fb7c21f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:18.465393 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-15e9158f-5105-4b4e-a46a-7db65fb7c21f None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:12:18.467657 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-15e9158f-5105-4b4e-a46a-7db65fb7c21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:12:18.573840 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e1215e-8424-4b85-a192-08bf2733b01f None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:18.597766 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 196/406] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:12:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:12:18.641876 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9158f-5105-4b4e-a46a-7db65fb7c21f None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:18.727898 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481b8705-dee8-433f-99cc-f542ed9a5986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:18.729051 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b8705-dee8-433f-99cc-f542ed9a5986 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:18.729553 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b8705-dee8-433f-99cc-f542ed9a5986 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:18.730380 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b8705-dee8-433f-99cc-f542ed9a5986 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:18.731827 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481b8705-dee8-433f-99cc-f542ed9a5986 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:18.732544 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481b8705-dee8-433f-99cc-f542ed9a5986 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:18.743886 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481b8705-dee8-433f-99cc-f542ed9a5986 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:18.759192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:18.772076 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 211/407] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:12:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 432 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:18.783164 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481b8705-dee8-433f-99cc-f542ed9a5986 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:18.808687 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9158f-5105-4b4e-a46a-7db65fb7c21f None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:18.893991 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e72e616-a5d2-409d-9c6c-0a46a722568c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:18.895470 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72e616-a5d2-409d-9c6c-0a46a722568c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:18.895978 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72e616-a5d2-409d-9c6c-0a46a722568c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:18.896486 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72e616-a5d2-409d-9c6c-0a46a722568c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:18.941526 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e72e616-a5d2-409d-9c6c-0a46a722568c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:12:18.964137 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-5e72e616-a5d2-409d-9c6c-0a46a722568c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:12:18.964137 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:18.964137 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:18.964137 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:18.964137 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:18.964137 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:18.964137 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:18.964137 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:18.964137 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:18.964137 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:18.964137 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:18.964137 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:18.964137 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:12:18.964137 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:12:18.964137 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:12:18.964137 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:12:18.966110 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:12:18.966110 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:12:18.966110 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:12:18.966110 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:12:18.966110 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:18.966110 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:18.966110 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:18.966110 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:18.966110 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:18.966110 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:18.966110 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:18.966110 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:18.966110 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:18.966110 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:18.966110 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:18.966110 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:18.967689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:18.967689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:18.967689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:18.967689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:18.967689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:12:18.967689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:12:18.967689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:12:18.967689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:12:18.967689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:12:18.967689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:12:18.967689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:12:18.967689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:12:18.967689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 197/408] 10.222.0.87 () {56 vars in 1096 bytes} [Tue Jun 21 18:12:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:12:18.970434 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676bef6-67cd-4491-9814-a2922fceaf96 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:18.970434 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676bef6-67cd-4491-9814-a2922fceaf96 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:18.977716 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676bef6-67cd-4491-9814-a2922fceaf96 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:18.978184 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 198/409] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:12:18 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:12:18.982020 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a676bef6-67cd-4491-9814-a2922fceaf96 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:19.006021 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:19.043287 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481b8705-dee8-433f-99cc-f542ed9a5986 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:19.109968 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cf62b5-0c27-4c49-a149-6aa6080c0733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:19.111629 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf62b5-0c27-4c49-a149-6aa6080c0733 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:19.112096 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf62b5-0c27-4c49-a149-6aa6080c0733 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:19.112531 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf62b5-0c27-4c49-a149-6aa6080c0733 None admin] PATH_INFO: `/v3/groups/b95be051ba524e2986ba6efc42648e76` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:19.115035 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cf62b5-0c27-4c49-a149-6aa6080c0733 None admin] RBAC: Authorizing `identity:get_group(group_id=b95be051ba524e2986ba6efc42648e76)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:19.118559 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cf62b5-0c27-4c49-a149-6aa6080c0733 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:19.123755 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 212/410] 10.222.0.87 () {56 vars in 1180 bytes} [Tue Jun 21 18:12:18 2022] GET /identity/v3/groups/b95be051ba524e2986ba6efc42648e76 => generated 226 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:19.127007 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f94dcc-b8be-4215-a773-8ab2556482cf None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:19.127493 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f94dcc-b8be-4215-a773-8ab2556482cf None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:19.127980 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f94dcc-b8be-4215-a773-8ab2556482cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:19.158711 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-09f94dcc-b8be-4215-a773-8ab2556482cf None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:12:19.161230 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-09f94dcc-b8be-4215-a773-8ab2556482cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:12:19.217166 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a676bef6-67cd-4491-9814-a2922fceaf96 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:19.276765 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 213/411] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:12:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:12:19.276765 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f94dcc-b8be-4215-a773-8ab2556482cf None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:19.288076 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:19.326904 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dffc166-cf06-4eed-b333-e198039abe9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:19.328257 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dffc166-cf06-4eed-b333-e198039abe9f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:19.328767 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dffc166-cf06-4eed-b333-e198039abe9f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:19.329235 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dffc166-cf06-4eed-b333-e198039abe9f None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:19.331270 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dffc166-cf06-4eed-b333-e198039abe9f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:19.331811 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dffc166-cf06-4eed-b333-e198039abe9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:19.334621 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dffc166-cf06-4eed-b333-e198039abe9f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:19.353940 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 199/412] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:12:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:12:19.366331 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dffc166-cf06-4eed-b333-e198039abe9f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:19.446460 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:19.515646 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f94dcc-b8be-4215-a773-8ab2556482cf None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:19.583077 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dffc166-cf06-4eed-b333-e198039abe9f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:19.594255 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd5664f-cc05-41bb-9b5e-2f276d2b4198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:19.596133 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd5664f-cc05-41bb-9b5e-2f276d2b4198 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:19.596888 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd5664f-cc05-41bb-9b5e-2f276d2b4198 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:19.598402 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd5664f-cc05-41bb-9b5e-2f276d2b4198 None admin] PATH_INFO: `/v3/projects/ee325f0b1b08455686f9e0a696ca57c8` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:19.600925 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd5664f-cc05-41bb-9b5e-2f276d2b4198 None admin] RBAC: Authorizing `identity:get_project(project_id=ee325f0b1b08455686f9e0a696ca57c8)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:19.604908 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd5664f-cc05-41bb-9b5e-2f276d2b4198 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:19.609425 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 214/413] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:12:19 2022] GET /identity/v3/projects/ee325f0b1b08455686f9e0a696ca57c8 => generated 302 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:19.611177 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd5664f-cc05-41bb-9b5e-2f276d2b4198 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:19.651625 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e04d083-081b-4442-be8d-afbc18cb0125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:19.653435 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04d083-081b-4442-be8d-afbc18cb0125 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:19.654191 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04d083-081b-4442-be8d-afbc18cb0125 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:19.654872 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04d083-081b-4442-be8d-afbc18cb0125 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:19.675805 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e04d083-081b-4442-be8d-afbc18cb0125 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:12:19.691558 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-3e04d083-081b-4442-be8d-afbc18cb0125 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:12:19.691558 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:19.691558 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:19.691558 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:19.691558 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:19.691558 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:19.691558 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:19.691558 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:19.691558 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:19.691558 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:19.691558 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:19.691558 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:19.691558 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:12:19.691558 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:12:19.691558 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:12:19.691558 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:12:19.693429 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:12:19.693429 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:12:19.693429 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:12:19.693429 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:12:19.693429 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:19.693429 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:19.693429 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:19.693429 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:19.693429 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:19.693429 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:19.693429 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:19.693429 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:19.693429 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:19.693429 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:19.693429 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:19.693429 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:19.695007 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:19.695007 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:19.695007 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:19.695007 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:19.695007 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:12:19.695007 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:12:19.695007 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:12:19.695007 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:12:19.695007 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:12:19.695007 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:12:19.695007 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:12:19.695007 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:12:19.705174 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 200/414] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:12:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:19.707122 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e04d083-081b-4442-be8d-afbc18cb0125 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:19.802126 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:19.822711 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd5664f-cc05-41bb-9b5e-2f276d2b4198 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:19.889851 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb0cbd9-9f20-4431-9018-a33fcdbc1a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:19.891521 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0cbd9-9f20-4431-9018-a33fcdbc1a2f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:19.892103 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0cbd9-9f20-4431-9018-a33fcdbc1a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:19.892692 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0cbd9-9f20-4431-9018-a33fcdbc1a2f None admin] PATH_INFO: `/v3/groups/d31a036caa574bbc943d9eabac35f5b2` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:19.894940 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb0cbd9-9f20-4431-9018-a33fcdbc1a2f None admin] RBAC: Authorizing `identity:get_group(group_id=d31a036caa574bbc943d9eabac35f5b2)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:19.897751 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb0cbd9-9f20-4431-9018-a33fcdbc1a2f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:19.907642 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 215/415] 10.222.0.87 () {56 vars in 1180 bytes} [Tue Jun 21 18:12:19 2022] GET /identity/v3/groups/d31a036caa574bbc943d9eabac35f5b2 => generated 229 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:19.907642 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb0cbd9-9f20-4431-9018-a33fcdbc1a2f None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:19.920175 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:19.950039 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e04d083-081b-4442-be8d-afbc18cb0125 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:20.018911 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad233007-fc39-4407-8406-0d1a187b60e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:20.020572 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad233007-fc39-4407-8406-0d1a187b60e0 None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:20.021159 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad233007-fc39-4407-8406-0d1a187b60e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:20.021763 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad233007-fc39-4407-8406-0d1a187b60e0 None admin] PATH_INFO: `/v3/projects/ee325f0b1b08455686f9e0a696ca57c8/groups/b95be051ba524e2986ba6efc42648e76/roles/46e77bb17fc3465496a7a3a29d210042` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:20.026253 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad233007-fc39-4407-8406-0d1a187b60e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=ee325f0b1b08455686f9e0a696ca57c8, group_id=b95be051ba524e2986ba6efc42648e76, role_id=46e77bb17fc3465496a7a3a29d210042)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:20.029683 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad233007-fc39-4407-8406-0d1a187b60e0 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:20.055473 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 201/416] 10.222.0.87 () {58 vars in 1442 bytes} [Tue Jun 21 18:12:19 2022] PUT /identity/v3/projects/ee325f0b1b08455686f9e0a696ca57c8/groups/b95be051ba524e2986ba6efc42648e76/roles/46e77bb17fc3465496a7a3a29d210042 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:12:20.057237 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad233007-fc39-4407-8406-0d1a187b60e0 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:20.133341 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb0cbd9-9f20-4431-9018-a33fcdbc1a2f None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:20.200121 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6ce511-9fe8-4260-aa82-c84941e2d7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:20.201263 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ce511-9fe8-4260-aa82-c84941e2d7a0 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:20.201903 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ce511-9fe8-4260-aa82-c84941e2d7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:20.202718 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ce511-9fe8-4260-aa82-c84941e2d7a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:20.203997 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6ce511-9fe8-4260-aa82-c84941e2d7a0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:20.204765 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6ce511-9fe8-4260-aa82-c84941e2d7a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:20.206455 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6ce511-9fe8-4260-aa82-c84941e2d7a0 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:20.218292 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 216/417] 10.222.0.87 () {56 vars in 1101 bytes} [Tue Jun 21 18:12:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:20.229483 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6ce511-9fe8-4260-aa82-c84941e2d7a0 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:20.291176 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad233007-fc39-4407-8406-0d1a187b60e0 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:20.309340 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:20.384741 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e512f1-e742-428c-af00-96ca5da56407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:20.390962 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e512f1-e742-428c-af00-96ca5da56407 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:20.390962 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e512f1-e742-428c-af00-96ca5da56407 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:20.390962 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e512f1-e742-428c-af00-96ca5da56407 None admin] PATH_INFO: `/v3/projects/58da051d3b4a47fab4c706fb517e9ae8` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:20.393720 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e512f1-e742-428c-af00-96ca5da56407 None admin] RBAC: Authorizing `identity:get_project(project_id=58da051d3b4a47fab4c706fb517e9ae8)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:20.396468 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e512f1-e742-428c-af00-96ca5da56407 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:20.401711 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 202/418] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:12:20 2022] GET /identity/v3/projects/58da051d3b4a47fab4c706fb517e9ae8 => generated 344 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:20.412016 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e512f1-e742-428c-af00-96ca5da56407 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:20.463881 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:20.546828 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6ce511-9fe8-4260-aa82-c84941e2d7a0 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:20.644325 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1d4a3c-e201-4bc2-a86d-69260306d9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:20.648199 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d4a3c-e201-4bc2-a86d-69260306d9a9 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:20.648661 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d4a3c-e201-4bc2-a86d-69260306d9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:20.649153 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d4a3c-e201-4bc2-a86d-69260306d9a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:20.712875 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e512f1-e742-428c-af00-96ca5da56407 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:20.819296 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1d4a3c-e201-4bc2-a86d-69260306d9a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:20.821703 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1d4a3c-e201-4bc2-a86d-69260306d9a9 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:20.832220 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57909ed-6a28-483b-b509-e6fed821b75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:20.833904 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57909ed-6a28-483b-b509-e6fed821b75e None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:20.834371 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57909ed-6a28-483b-b509-e6fed821b75e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:20.834866 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57909ed-6a28-483b-b509-e6fed821b75e None admin] PATH_INFO: `/v3/projects/58da051d3b4a47fab4c706fb517e9ae8/groups/d31a036caa574bbc943d9eabac35f5b2/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:20.838857 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57909ed-6a28-483b-b509-e6fed821b75e None admin] RBAC: Authorizing `identity:create_grant(project_id=58da051d3b4a47fab4c706fb517e9ae8, group_id=d31a036caa574bbc943d9eabac35f5b2, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:20.841478 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57909ed-6a28-483b-b509-e6fed821b75e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:20.866749 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 203/419] 10.222.0.87 () {58 vars in 1442 bytes} [Tue Jun 21 18:12:20 2022] PUT /identity/v3/projects/58da051d3b4a47fab4c706fb517e9ae8/groups/d31a036caa574bbc943d9eabac35f5b2/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:12:20.892243 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfc683-c842-41d7-88cd-74710133d4a7 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:20.892647 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfc683-c842-41d7-88cd-74710133d4a7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:20.892837 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfc683-c842-41d7-88cd-74710133d4a7 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:20.895130 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 204/420] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:12:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:12:20.902209 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e149b8-e16a-41a0-8dcf-c07effd268d9 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:20.902591 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e149b8-e16a-41a0-8dcf-c07effd268d9 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:20.902773 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e149b8-e16a-41a0-8dcf-c07effd268d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:20.948898 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b9e149b8-e16a-41a0-8dcf-c07effd268d9 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:12:20.956908 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b9e149b8-e16a-41a0-8dcf-c07effd268d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:12:20.985935 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:21.029253 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:21.111139 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 217/421] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:12:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 882 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 21 18:12:21.119847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1d4a3c-e201-4bc2-a86d-69260306d9a9 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:21.145521 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 205/422] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:12:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:12:21.182367 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e149b8-e16a-41a0-8dcf-c07effd268d9 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:21.339220 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e149b8-e16a-41a0-8dcf-c07effd268d9 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:21.372472 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1d4a3c-e201-4bc2-a86d-69260306d9a9 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:21.422764 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b41c7e-e0f8-4b24-bd5a-c36a708161dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:21.424944 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b41c7e-e0f8-4b24-bd5a-c36a708161dd None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:21.425411 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b41c7e-e0f8-4b24-bd5a-c36a708161dd None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:21.425890 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b41c7e-e0f8-4b24-bd5a-c36a708161dd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:21.446547 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc469b-e043-4689-b044-efc4b0155f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:21.448103 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc469b-e043-4689-b044-efc4b0155f1e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:21.448576 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc469b-e043-4689-b044-efc4b0155f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:21.449075 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc469b-e043-4689-b044-efc4b0155f1e None admin] PATH_INFO: `/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:21.449471 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b41c7e-e0f8-4b24-bd5a-c36a708161dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:12:21.451502 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bc469b-e043-4689-b044-efc4b0155f1e None admin] RBAC: Authorizing `identity:get_project(project_id=48caa0e9d7fe40d28e99abd1a7a6896f)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:21.454481 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bc469b-e043-4689-b044-efc4b0155f1e None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:21.462220 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 218/423] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:12:21 2022] GET /identity/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f => generated 298 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:21.464727 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6993398f-89f3-428b-ad7c-d79b0b6471a4 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:21.465700 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6993398f-89f3-428b-ad7c-d79b0b6471a4 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:21.465862 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-79b41c7e-e0f8-4b24-bd5a-c36a708161dd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:12:21.465862 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:21.465862 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:21.465862 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:21.465862 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:21.465862 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:21.465862 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:21.465862 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:21.465862 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:21.465862 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:21.465862 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:21.465862 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:21.465862 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:12:21.465862 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:12:21.465862 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:12:21.465862 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:12:21.467440 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:12:21.467440 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:12:21.467440 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:12:21.467440 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:12:21.467440 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:21.467440 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:21.467440 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:21.467440 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:21.467440 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:21.467440 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:21.467440 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:21.467440 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:21.467440 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:21.467440 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:21.467440 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:21.467440 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:21.469002 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:21.469002 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:21.469002 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:21.469002 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:21.469002 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:12:21.469002 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:12:21.469002 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:12:21.469002 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:12:21.469002 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:12:21.469002 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:12:21.469002 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:12:21.469002 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:12:21.470180 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6993398f-89f3-428b-ad7c-d79b0b6471a4 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:21.470304 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 206/424] 10.222.0.87 () {56 vars in 1114 bytes} [Tue Jun 21 18:12:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:12:21.474078 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 219/425] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:12:21 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:12:21.474375 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b41c7e-e0f8-4b24-bd5a-c36a708161dd None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:21.477813 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6993398f-89f3-428b-ad7c-d79b0b6471a4 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:21.545549 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:21.566300 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:21.671130 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b41c7e-e0f8-4b24-bd5a-c36a708161dd None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:21.674435 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6993398f-89f3-428b-ad7c-d79b0b6471a4 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:21.715004 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3037eb-b6c1-4133-9dd4-0a3633357e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:21.715909 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3037eb-b6c1-4133-9dd4-0a3633357e63 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:21.716192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3037eb-b6c1-4133-9dd4-0a3633357e63 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:21.716468 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3037eb-b6c1-4133-9dd4-0a3633357e63 None admin] PATH_INFO: `/v3/groups/b95be051ba524e2986ba6efc42648e76` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:21.717935 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3037eb-b6c1-4133-9dd4-0a3633357e63 None admin] RBAC: Authorizing `identity:get_group(group_id=b95be051ba524e2986ba6efc42648e76)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:21.719613 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3037eb-b6c1-4133-9dd4-0a3633357e63 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:21.722556 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 207/426] 10.222.0.87 () {56 vars in 1180 bytes} [Tue Jun 21 18:12:21 2022] GET /identity/v3/groups/b95be051ba524e2986ba6efc42648e76 => generated 226 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:21.724690 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73e355-f67e-48ab-bb61-8149fbec651c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:21.725155 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73e355-f67e-48ab-bb61-8149fbec651c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:21.725627 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73e355-f67e-48ab-bb61-8149fbec651c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:21.742132 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eafbeb-a137-4abe-9621-ff05cc358412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:21.743693 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eafbeb-a137-4abe-9621-ff05cc358412 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:21.744405 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eafbeb-a137-4abe-9621-ff05cc358412 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:21.745121 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eafbeb-a137-4abe-9621-ff05cc358412 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:21.746728 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eafbeb-a137-4abe-9621-ff05cc358412 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:21.747574 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eafbeb-a137-4abe-9621-ff05cc358412 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:21.750965 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eafbeb-a137-4abe-9621-ff05cc358412 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:21.755212 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-7d73e355-f67e-48ab-bb61-8149fbec651c None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:12:21.756610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7d73e355-f67e-48ab-bb61-8149fbec651c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:12:21.772578 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 220/427] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:12:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:21.774508 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eafbeb-a137-4abe-9621-ff05cc358412 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:21.861233 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 208/428] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:12:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:12:21.862208 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d73e355-f67e-48ab-bb61-8149fbec651c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:21.894738 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:21.979143 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eafbeb-a137-4abe-9621-ff05cc358412 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:21.999775 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:22.049273 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba360e3a-e06f-4dd5-9ee7-6f1f34ce7a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:22.050104 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba360e3a-e06f-4dd5-9ee7-6f1f34ce7a7d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:22.050389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba360e3a-e06f-4dd5-9ee7-6f1f34ce7a7d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:22.050665 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba360e3a-e06f-4dd5-9ee7-6f1f34ce7a7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:22.051728 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba360e3a-e06f-4dd5-9ee7-6f1f34ce7a7d None admin] RBAC: Adding query filter params (group.id=b95be051ba524e2986ba6efc42648e76, role.id=da6d4a0ab8c04809ba120170fb4ae8a9, scope.project.id=48caa0e9d7fe40d28e99abd1a7a6896f) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:22.052045 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba360e3a-e06f-4dd5-9ee7-6f1f34ce7a7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:22.053858 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba360e3a-e06f-4dd5-9ee7-6f1f34ce7a7d None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:22.064890 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d73e355-f67e-48ab-bb61-8149fbec651c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:22.066421 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 221/429] 10.222.0.87 () {56 vars in 1376 bytes} [Tue Jun 21 18:12:21 2022] GET /identity/v3/role_assignments?group.id=b95be051ba524e2986ba6efc42648e76&scope.project.id=48caa0e9d7fe40d28e99abd1a7a6896f&role.id=da6d4a0ab8c04809ba120170fb4ae8a9 => generated 611 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:22.067999 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba360e3a-e06f-4dd5-9ee7-6f1f34ce7a7d None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:22.211129 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca56f8b1-e0eb-4ad8-82eb-c33df7270d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:22.212328 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56f8b1-e0eb-4ad8-82eb-c33df7270d38 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:22.212800 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56f8b1-e0eb-4ad8-82eb-c33df7270d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:22.213371 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56f8b1-e0eb-4ad8-82eb-c33df7270d38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:22.273562 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba360e3a-e06f-4dd5-9ee7-6f1f34ce7a7d None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:22.345355 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b8cfcb-be43-47c2-993c-697e232325b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:22.346759 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8cfcb-be43-47c2-993c-697e232325b9 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:22.347587 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8cfcb-be43-47c2-993c-697e232325b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:22.348012 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8cfcb-be43-47c2-993c-697e232325b9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:22.370541 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b8cfcb-be43-47c2-993c-697e232325b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:12:22.386818 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-65b8cfcb-be43-47c2-993c-697e232325b9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:12:22.386818 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:22.386818 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:22.386818 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:22.386818 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:22.386818 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:22.386818 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:22.386818 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:22.386818 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:22.386818 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:22.386818 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:22.386818 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:22.386818 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:12:22.386818 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:12:22.386818 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:12:22.386818 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:12:22.387797 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:12:22.387797 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:12:22.387797 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:12:22.387797 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:12:22.387797 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:22.387797 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:22.387797 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:22.387797 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:22.387797 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:22.387797 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:22.387797 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:22.387797 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:22.387797 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:22.387797 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:22.387797 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:22.387797 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:22.388656 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:22.388656 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:22.388656 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:22.388656 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:22.388656 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:12:22.388656 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:12:22.388656 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:12:22.388656 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:12:22.388656 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:12:22.388656 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:12:22.388656 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:12:22.388656 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:12:22.389944 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 222/430] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:12:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:22.399690 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b8cfcb-be43-47c2-993c-697e232325b9 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:22.466350 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca56f8b1-e0eb-4ad8-82eb-c33df7270d38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:22.469583 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca56f8b1-e0eb-4ad8-82eb-c33df7270d38 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:22.471774 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:22.533723 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:22.664340 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b8cfcb-be43-47c2-993c-697e232325b9 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:22.704995 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 209/431] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:12:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 843 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:12:22.713710 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca56f8b1-e0eb-4ad8-82eb-c33df7270d38 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:22.761168 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f3bf9-78b8-4cee-9969-52f7012acb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:22.762746 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f3bf9-78b8-4cee-9969-52f7012acb82 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:22.763245 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f3bf9-78b8-4cee-9969-52f7012acb82 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:22.763733 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f3bf9-78b8-4cee-9969-52f7012acb82 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:22.767729 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8f3bf9-78b8-4cee-9969-52f7012acb82 None admin] RBAC: Adding query filter params (name=member) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:22.767729 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8f3bf9-78b8-4cee-9969-52f7012acb82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:22.768972 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8f3bf9-78b8-4cee-9969-52f7012acb82 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:22.792860 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 223/432] 10.222.0.87 () {56 vars in 1101 bytes} [Tue Jun 21 18:12:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:22.809759 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f3bf9-78b8-4cee-9969-52f7012acb82 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:22.960718 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca56f8b1-e0eb-4ad8-82eb-c33df7270d38 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:23.010066 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:23.038846 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312e6b58-8dd3-44d3-a9d7-9097a05b6b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:23.040185 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e6b58-8dd3-44d3-a9d7-9097a05b6b82 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:23.040628 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e6b58-8dd3-44d3-a9d7-9097a05b6b82 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:23.041071 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e6b58-8dd3-44d3-a9d7-9097a05b6b82 None admin] PATH_INFO: `/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:23.045909 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312e6b58-8dd3-44d3-a9d7-9097a05b6b82 None admin] RBAC: Authorizing `identity:get_project(project_id=48caa0e9d7fe40d28e99abd1a7a6896f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:23.047958 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312e6b58-8dd3-44d3-a9d7-9097a05b6b82 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:23.051347 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 210/433] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:12:22 2022] GET /identity/v3/projects/48caa0e9d7fe40d28e99abd1a7a6896f => generated 298 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:12:23.057254 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312e6b58-8dd3-44d3-a9d7-9097a05b6b82 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:23.065717 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:23.094829 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f3bf9-78b8-4cee-9969-52f7012acb82 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:23.146548 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df758a2-80df-4869-92cd-1a3f579187ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:23.147914 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df758a2-80df-4869-92cd-1a3f579187ef None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:23.148382 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df758a2-80df-4869-92cd-1a3f579187ef None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:23.148807 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df758a2-80df-4869-92cd-1a3f579187ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:23.243982 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312e6b58-8dd3-44d3-a9d7-9097a05b6b82 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:23.277940 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df758a2-80df-4869-92cd-1a3f579187ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:23.281437 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df758a2-80df-4869-92cd-1a3f579187ef None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:23.301933 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90013add-400e-42e0-944f-29e0ca05437f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:23.302877 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90013add-400e-42e0-944f-29e0ca05437f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:23.303281 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90013add-400e-42e0-944f-29e0ca05437f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:23.303655 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90013add-400e-42e0-944f-29e0ca05437f None admin] PATH_INFO: `/v3/groups/b95be051ba524e2986ba6efc42648e76` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:23.305015 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90013add-400e-42e0-944f-29e0ca05437f None admin] RBAC: Authorizing `identity:get_group(group_id=b95be051ba524e2986ba6efc42648e76)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:23.306636 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90013add-400e-42e0-944f-29e0ca05437f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:23.308558 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 211/434] 10.222.0.87 () {56 vars in 1180 bytes} [Tue Jun 21 18:12:23 2022] GET /identity/v3/groups/b95be051ba524e2986ba6efc42648e76 => generated 226 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:23.318768 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90013add-400e-42e0-944f-29e0ca05437f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:23.394653 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:23.407569 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:23.429625 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 224/435] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:12:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 622 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 21 18:12:23.436393 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df758a2-80df-4869-92cd-1a3f579187ef None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:23.459578 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90013add-400e-42e0-944f-29e0ca05437f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:23.511559 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d6572-ef63-481e-b1c9-e221585b4bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:23.512268 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d6572-ef63-481e-b1c9-e221585b4bc7 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:23.513214 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d6572-ef63-481e-b1c9-e221585b4bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:23.513214 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d6572-ef63-481e-b1c9-e221585b4bc7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:23.513709 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946d6572-ef63-481e-b1c9-e221585b4bc7 None admin] RBAC: Adding query filter params (group.id=b95be051ba524e2986ba6efc42648e76, role.id=46e77bb17fc3465496a7a3a29d210042, scope.project.id=48caa0e9d7fe40d28e99abd1a7a6896f) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:23.514099 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946d6572-ef63-481e-b1c9-e221585b4bc7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:23.515771 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946d6572-ef63-481e-b1c9-e221585b4bc7 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:23.523114 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 212/436] 10.222.0.87 () {56 vars in 1376 bytes} [Tue Jun 21 18:12:23 2022] GET /identity/v3/role_assignments?group.id=b95be051ba524e2986ba6efc42648e76&scope.project.id=48caa0e9d7fe40d28e99abd1a7a6896f&role.id=46e77bb17fc3465496a7a3a29d210042 => generated 611 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:23.582297 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df758a2-80df-4869-92cd-1a3f579187ef None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:23.630470 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce418c-35d5-4412-974a-5d78bb552b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:23.631532 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce418c-35d5-4412-974a-5d78bb552b0a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:23.631825 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce418c-35d5-4412-974a-5d78bb552b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:23.632111 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce418c-35d5-4412-974a-5d78bb552b0a None admin] PATH_INFO: `/v3/projects/ee325f0b1b08455686f9e0a696ca57c8` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:23.633591 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce418c-35d5-4412-974a-5d78bb552b0a None admin] RBAC: Authorizing `identity:get_project(project_id=ee325f0b1b08455686f9e0a696ca57c8)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:23.635162 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce418c-35d5-4412-974a-5d78bb552b0a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:23.637387 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 225/437] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:12:23 2022] GET /identity/v3/projects/ee325f0b1b08455686f9e0a696ca57c8 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:23.645997 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d6572-ef63-481e-b1c9-e221585b4bc7 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:23.693544 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40101d-d282-4a96-a834-6d7ae247db40 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:23.694629 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40101d-d282-4a96-a834-6d7ae247db40 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:23.695570 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40101d-d282-4a96-a834-6d7ae247db40 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:23.699015 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 226/438] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:12:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:12:23.708692 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8a817-20d7-4e9e-9297-6e62d2a72361 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:23.709621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8a817-20d7-4e9e-9297-6e62d2a72361 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:23.710472 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8a817-20d7-4e9e-9297-6e62d2a72361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:23.742796 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-c7a8a817-20d7-4e9e-9297-6e62d2a72361 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:12:23.744504 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c7a8a817-20d7-4e9e-9297-6e62d2a72361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:12:23.821948 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d6572-ef63-481e-b1c9-e221585b4bc7 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:23.859346 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:23.872548 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:23.880972 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cfb290-affe-4dac-8e8c-873f10656156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:23.882910 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cfb290-affe-4dac-8e8c-873f10656156 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:23.883843 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cfb290-affe-4dac-8e8c-873f10656156 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:23.885067 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cfb290-affe-4dac-8e8c-873f10656156 None admin] PATH_INFO: `/v3/groups/b95be051ba524e2986ba6efc42648e76` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:23.889250 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cfb290-affe-4dac-8e8c-873f10656156 None admin] RBAC: Authorizing `identity:get_group(group_id=b95be051ba524e2986ba6efc42648e76)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:23.891218 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cfb290-affe-4dac-8e8c-873f10656156 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:23.894839 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 213/439] 10.222.0.87 () {56 vars in 1180 bytes} [Tue Jun 21 18:12:23 2022] GET /identity/v3/groups/b95be051ba524e2986ba6efc42648e76 => generated 226 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:23.903123 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cfb290-affe-4dac-8e8c-873f10656156 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:23.911871 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 227/440] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:12:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:12:23.943708 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a8a817-20d7-4e9e-9297-6e62d2a72361 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:24.091739 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cfb290-affe-4dac-8e8c-873f10656156 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:24.133022 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a8a817-20d7-4e9e-9297-6e62d2a72361 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:24.141678 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa541c65-6e77-416f-867c-323f8b88d988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:24.142679 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa541c65-6e77-416f-867c-323f8b88d988 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:24.143486 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa541c65-6e77-416f-867c-323f8b88d988 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:24.143486 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa541c65-6e77-416f-867c-323f8b88d988 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:24.144732 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa541c65-6e77-416f-867c-323f8b88d988 None admin] RBAC: Adding query filter params (group.id=b95be051ba524e2986ba6efc42648e76, role.id=da6d4a0ab8c04809ba120170fb4ae8a9, scope.project.id=ee325f0b1b08455686f9e0a696ca57c8) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:24.145265 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa541c65-6e77-416f-867c-323f8b88d988 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:24.147203 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa541c65-6e77-416f-867c-323f8b88d988 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:24.154802 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 214/441] 10.222.0.87 () {56 vars in 1376 bytes} [Tue Jun 21 18:12:23 2022] GET /identity/v3/role_assignments?group.id=b95be051ba524e2986ba6efc42648e76&scope.project.id=ee325f0b1b08455686f9e0a696ca57c8&role.id=da6d4a0ab8c04809ba120170fb4ae8a9 => generated 611 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:12:24.158194 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1da722-9356-4272-826b-925f4f85dfe2 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:24.158641 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1da722-9356-4272-826b-925f4f85dfe2 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:24.159084 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1da722-9356-4272-826b-925f4f85dfe2 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:24.161467 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 215/442] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:12:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:12:24.170530 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19333d-d9f9-4204-a02c-4c605daaab02 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:24.171033 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19333d-d9f9-4204-a02c-4c605daaab02 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:24.172750 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19333d-d9f9-4204-a02c-4c605daaab02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:24.183217 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7912348-d78b-4c42-8eb4-3249fe984e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:24.184065 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7912348-d78b-4c42-8eb4-3249fe984e15 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:24.184449 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7912348-d78b-4c42-8eb4-3249fe984e15 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:24.184665 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7912348-d78b-4c42-8eb4-3249fe984e15 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:24.200363 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-2c19333d-d9f9-4204-a02c-4c605daaab02 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:12:24.201678 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7912348-d78b-4c42-8eb4-3249fe984e15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:12:24.202321 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2c19333d-d9f9-4204-a02c-4c605daaab02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:12:24.211409 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-a7912348-d78b-4c42-8eb4-3249fe984e15 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:12:24.211409 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:24.211409 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:24.211409 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:24.211409 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:24.211409 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:24.211409 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:24.211409 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:24.211409 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:24.211409 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:24.211409 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:24.211409 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:24.211409 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:12:24.211409 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:12:24.211409 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:12:24.211409 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:12:24.212621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:12:24.212621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:12:24.212621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:12:24.212621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:12:24.212621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:24.212621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:24.212621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:24.212621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:24.212621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:24.212621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:24.212621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:24.212621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:24.212621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:24.212621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:24.212621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:24.212621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:24.213423 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:24.213423 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:24.213423 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:24.213423 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:24.213423 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:12:24.213423 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:12:24.213423 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:12:24.213423 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:12:24.213423 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:12:24.213423 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:12:24.213423 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:12:24.213423 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:12:24.213423 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 228/443] 10.222.0.87 () {56 vars in 1114 bytes} [Tue Jun 21 18:12:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:12:24.219812 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7912348-d78b-4c42-8eb4-3249fe984e15 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:24.281309 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:24.283318 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:24.309184 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 216/444] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:12:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:12:24.334317 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19333d-d9f9-4204-a02c-4c605daaab02 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:24.384878 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7912348-d78b-4c42-8eb4-3249fe984e15 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:24.438489 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a953e2e-a307-4572-b78a-1a793597a84d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:24.439706 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a953e2e-a307-4572-b78a-1a793597a84d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:24.439990 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a953e2e-a307-4572-b78a-1a793597a84d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:24.440270 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a953e2e-a307-4572-b78a-1a793597a84d None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:24.441182 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a953e2e-a307-4572-b78a-1a793597a84d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:24.441544 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a953e2e-a307-4572-b78a-1a793597a84d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:24.443158 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a953e2e-a307-4572-b78a-1a793597a84d None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:24.453789 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 229/445] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:12:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:12:24.459280 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a953e2e-a307-4572-b78a-1a793597a84d None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:24.478628 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19333d-d9f9-4204-a02c-4c605daaab02 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:24.544539 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff04c6f8-0e89-4648-9bad-17643ea15f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:24.546086 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff04c6f8-0e89-4648-9bad-17643ea15f64 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:24.546587 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff04c6f8-0e89-4648-9bad-17643ea15f64 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:24.547039 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff04c6f8-0e89-4648-9bad-17643ea15f64 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:24.564727 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff04c6f8-0e89-4648-9bad-17643ea15f64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:12:24.576375 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-ff04c6f8-0e89-4648-9bad-17643ea15f64 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:12:24.576375 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:24.576375 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:24.576375 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:24.576375 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:24.576375 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:24.576375 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:24.576375 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:24.576375 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:24.576375 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:24.576375 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:24.576375 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:24.576375 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:12:24.576375 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:12:24.576375 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:12:24.576375 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:12:24.577540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:12:24.577540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:12:24.577540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:12:24.577540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:12:24.577540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:24.577540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:24.577540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:24.577540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:24.577540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:24.577540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:24.577540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:24.577540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:24.577540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:24.577540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:24.577540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:24.577540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:24.578417 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:24.578417 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:24.578417 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:24.578417 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:24.578417 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:12:24.578417 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:12:24.578417 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:12:24.578417 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:12:24.578417 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:12:24.578417 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:12:24.578417 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:12:24.578417 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:12:24.579133 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 217/446] 10.222.0.87 () {56 vars in 1096 bytes} [Tue Jun 21 18:12:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:24.587189 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff04c6f8-0e89-4648-9bad-17643ea15f64 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:24.635476 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a953e2e-a307-4572-b78a-1a793597a84d None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:24.639229 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:24.656273 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:24.698714 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1c9d94-ce64-44d1-bf19-d9df76de7032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:24.699450 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1c9d94-ce64-44d1-bf19-d9df76de7032 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:24.699884 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1c9d94-ce64-44d1-bf19-d9df76de7032 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:24.700183 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1c9d94-ce64-44d1-bf19-d9df76de7032 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:24.736457 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff04c6f8-0e89-4648-9bad-17643ea15f64 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:24.789804 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1c9d94-ce64-44d1-bf19-d9df76de7032 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:24.791436 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1c9d94-ce64-44d1-bf19-d9df76de7032 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:24.804894 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb9d89-a2a2-4c61-8bf9-e320006408f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:24.805950 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb9d89-a2a2-4c61-8bf9-e320006408f7 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:24.806401 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb9d89-a2a2-4c61-8bf9-e320006408f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:24.806857 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb9d89-a2a2-4c61-8bf9-e320006408f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:24.808130 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcb9d89-a2a2-4c61-8bf9-e320006408f7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:24.808731 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcb9d89-a2a2-4c61-8bf9-e320006408f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:24.810565 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcb9d89-a2a2-4c61-8bf9-e320006408f7 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:24.823562 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 218/447] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:12:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:12:24.830683 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb9d89-a2a2-4c61-8bf9-e320006408f7 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:24.958058 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 230/448] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:12:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 21 18:12:24.959482 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1c9d94-ce64-44d1-bf19-d9df76de7032 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:24.963201 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:24.982123 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb9d89-a2a2-4c61-8bf9-e320006408f7 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:24.992161 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:25.023682 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1248ad41-dbdc-406e-9a02-3f52f90a06ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:25.024508 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1248ad41-dbdc-406e-9a02-3f52f90a06ae None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:25.024897 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1248ad41-dbdc-406e-9a02-3f52f90a06ae None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:25.025239 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1248ad41-dbdc-406e-9a02-3f52f90a06ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:25.085503 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1c9d94-ce64-44d1-bf19-d9df76de7032 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:25.124362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1248ad41-dbdc-406e-9a02-3f52f90a06ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:25.126376 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1248ad41-dbdc-406e-9a02-3f52f90a06ae None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:25.137473 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51fc1ef-46dc-4948-8282-27c8ce1ff7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:25.141314 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51fc1ef-46dc-4948-8282-27c8ce1ff7e0 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:25.141314 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51fc1ef-46dc-4948-8282-27c8ce1ff7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:25.141314 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51fc1ef-46dc-4948-8282-27c8ce1ff7e0 None admin] PATH_INFO: `/v3/projects/ee325f0b1b08455686f9e0a696ca57c8` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:25.147805 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51fc1ef-46dc-4948-8282-27c8ce1ff7e0 None admin] RBAC: Authorizing `identity:get_project(project_id=ee325f0b1b08455686f9e0a696ca57c8)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:25.147805 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51fc1ef-46dc-4948-8282-27c8ce1ff7e0 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:25.148339 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 231/449] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:12:24 2022] GET /identity/v3/projects/ee325f0b1b08455686f9e0a696ca57c8 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:25.155623 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51fc1ef-46dc-4948-8282-27c8ce1ff7e0 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:25.268073 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:25.288383 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 219/450] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:12:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 21 18:12:25.295197 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1248ad41-dbdc-406e-9a02-3f52f90a06ae None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:25.306856 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:25.307803 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51fc1ef-46dc-4948-8282-27c8ce1ff7e0 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:25.361782 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95943f52-cae0-4b51-bb83-756447e611cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:25.363206 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95943f52-cae0-4b51-bb83-756447e611cd None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:25.363690 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95943f52-cae0-4b51-bb83-756447e611cd None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:25.364047 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95943f52-cae0-4b51-bb83-756447e611cd None admin] PATH_INFO: `/v3/groups/b95be051ba524e2986ba6efc42648e76` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:25.366097 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95943f52-cae0-4b51-bb83-756447e611cd None admin] RBAC: Authorizing `identity:get_group(group_id=b95be051ba524e2986ba6efc42648e76)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:25.368936 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95943f52-cae0-4b51-bb83-756447e611cd None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:25.372291 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 232/451] 10.222.0.87 () {56 vars in 1180 bytes} [Tue Jun 21 18:12:25 2022] GET /identity/v3/groups/b95be051ba524e2986ba6efc42648e76 => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:25.382257 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95943f52-cae0-4b51-bb83-756447e611cd None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:25.458708 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1248ad41-dbdc-406e-9a02-3f52f90a06ae None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:25.506079 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d602e5-3841-41f1-887d-29bb7cefefd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:25.507106 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d602e5-3841-41f1-887d-29bb7cefefd7 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:25.507511 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d602e5-3841-41f1-887d-29bb7cefefd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:25.507959 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d602e5-3841-41f1-887d-29bb7cefefd7 None admin] PATH_INFO: `/v3/projects/58da051d3b4a47fab4c706fb517e9ae8` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:25.509436 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d602e5-3841-41f1-887d-29bb7cefefd7 None admin] RBAC: Authorizing `identity:get_project(project_id=58da051d3b4a47fab4c706fb517e9ae8)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:25.511092 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d602e5-3841-41f1-887d-29bb7cefefd7 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:25.513161 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 220/452] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:12:25 2022] GET /identity/v3/projects/58da051d3b4a47fab4c706fb517e9ae8 => generated 344 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:12:25.520428 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d602e5-3841-41f1-887d-29bb7cefefd7 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:25.584871 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95943f52-cae0-4b51-bb83-756447e611cd None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:25.645926 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff75a8c0-17d0-4688-8b20-0fb48936bc26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:25.647157 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff75a8c0-17d0-4688-8b20-0fb48936bc26 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:25.647619 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff75a8c0-17d0-4688-8b20-0fb48936bc26 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:25.648063 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff75a8c0-17d0-4688-8b20-0fb48936bc26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:25.649328 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff75a8c0-17d0-4688-8b20-0fb48936bc26 None admin] RBAC: Adding query filter params (group.id=b95be051ba524e2986ba6efc42648e76, role.id=46e77bb17fc3465496a7a3a29d210042, scope.project.id=ee325f0b1b08455686f9e0a696ca57c8) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:25.649724 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff75a8c0-17d0-4688-8b20-0fb48936bc26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:25.651279 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff75a8c0-17d0-4688-8b20-0fb48936bc26 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:25.662471 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 233/453] 10.222.0.87 () {56 vars in 1376 bytes} [Tue Jun 21 18:12:25 2022] GET /identity/v3/role_assignments?group.id=b95be051ba524e2986ba6efc42648e76&scope.project.id=ee325f0b1b08455686f9e0a696ca57c8&role.id=46e77bb17fc3465496a7a3a29d210042 => generated 611 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:12:25.670985 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:25.719784 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d602e5-3841-41f1-887d-29bb7cefefd7 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:25.788534 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a6a606-fdc0-4eb2-a6d8-ed58b585b5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:25.790085 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6a606-fdc0-4eb2-a6d8-ed58b585b5c0 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:25.790580 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6a606-fdc0-4eb2-a6d8-ed58b585b5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:25.791072 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6a606-fdc0-4eb2-a6d8-ed58b585b5c0 None admin] PATH_INFO: `/v3/groups/d31a036caa574bbc943d9eabac35f5b2` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:25.793552 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a6a606-fdc0-4eb2-a6d8-ed58b585b5c0 None admin] RBAC: Authorizing `identity:get_group(group_id=d31a036caa574bbc943d9eabac35f5b2)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:25.796486 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a6a606-fdc0-4eb2-a6d8-ed58b585b5c0 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:25.799897 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 221/454] 10.222.0.87 () {56 vars in 1180 bytes} [Tue Jun 21 18:12:25 2022] GET /identity/v3/groups/d31a036caa574bbc943d9eabac35f5b2 => generated 229 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:12:25.810233 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff75a8c0-17d0-4688-8b20-0fb48936bc26 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:25.817854 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:26.007271 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff75a8c0-17d0-4688-8b20-0fb48936bc26 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:26.055369 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1904316c-dc23-4ea8-8683-ae925a4188bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:26.056541 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904316c-dc23-4ea8-8683-ae925a4188bf None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:26.056541 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904316c-dc23-4ea8-8683-ae925a4188bf None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:26.056541 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904316c-dc23-4ea8-8683-ae925a4188bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:26.057784 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1904316c-dc23-4ea8-8683-ae925a4188bf None admin] RBAC: Adding query filter params (group.id=d31a036caa574bbc943d9eabac35f5b2, role.id=98f0943a610f4a9e9f8de54f6e9f717d, scope.project.id=58da051d3b4a47fab4c706fb517e9ae8) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:26.058105 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1904316c-dc23-4ea8-8683-ae925a4188bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:26.059736 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1904316c-dc23-4ea8-8683-ae925a4188bf None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:26.069366 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 234/455] 10.222.0.87 () {56 vars in 1376 bytes} [Tue Jun 21 18:12:25 2022] GET /identity/v3/role_assignments?group.id=d31a036caa574bbc943d9eabac35f5b2&scope.project.id=58da051d3b4a47fab4c706fb517e9ae8&role.id=98f0943a610f4a9e9f8de54f6e9f717d => generated 611 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:29.260192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d959f3-0e3c-4f18-a872-b73b6362026e None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:29.261180 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d959f3-0e3c-4f18-a872-b73b6362026e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:29.261668 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d959f3-0e3c-4f18-a872-b73b6362026e None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:29.264287 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 222/456] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:12:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:12:29.276225 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f8e989-10b3-4484-bf3e-2a4a37ee6a74 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:29.277171 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f8e989-10b3-4484-bf3e-2a4a37ee6a74 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:29.277671 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f8e989-10b3-4484-bf3e-2a4a37ee6a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:29.311817 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-59f8e989-10b3-4484-bf3e-2a4a37ee6a74 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:12:29.313580 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-59f8e989-10b3-4484-bf3e-2a4a37ee6a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:12:29.448152 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:29.462183 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 235/457] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:12:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:12:29.511184 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d959f3-0e3c-4f18-a872-b73b6362026e None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:29.772663 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d959f3-0e3c-4f18-a872-b73b6362026e None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:29.864525 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eabc89-f385-4f09-80ec-5aef58224842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:29.866377 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eabc89-f385-4f09-80ec-5aef58224842 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:29.867186 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eabc89-f385-4f09-80ec-5aef58224842 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:29.867932 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eabc89-f385-4f09-80ec-5aef58224842 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:29.894975 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eabc89-f385-4f09-80ec-5aef58224842 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:29.896903 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eabc89-f385-4f09-80ec-5aef58224842 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:29.901252 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:29.934816 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-a5eabc89-f385-4f09-80ec-5aef58224842 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:29.934816 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:29.934816 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:29.934816 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:29.934816 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:29.934816 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:29.934816 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:29.934816 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:29.934816 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:29.934816 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:29.934816 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:29.934816 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:29.934816 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:12:29.934816 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:12:29.934816 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:12:29.936003 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:12:29.936003 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:29.936003 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:29.936003 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:29.936003 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:29.936003 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:29.936003 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:29.936003 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:29.936003 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:29.936003 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:29.936003 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:29.936003 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:29.936003 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:29.936003 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:29.936003 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:29.936003 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:29.936003 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:29.937355 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:12:29.937355 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:12:29.937355 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:29.937355 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:12:29.940024 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 223/458] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:12:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 430 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:29.946953 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f8e989-10b3-4484-bf3e-2a4a37ee6a74 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:30.157146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f8e989-10b3-4484-bf3e-2a4a37ee6a74 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:30.229322 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a4b786-05a8-41df-b447-25ae674c80e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:30.230527 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4b786-05a8-41df-b447-25ae674c80e8 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:30.231067 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4b786-05a8-41df-b447-25ae674c80e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:30.231556 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4b786-05a8-41df-b447-25ae674c80e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:30.232604 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a4b786-05a8-41df-b447-25ae674c80e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:30.233161 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a4b786-05a8-41df-b447-25ae674c80e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:30.234905 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a4b786-05a8-41df-b447-25ae674c80e8 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:30.255787 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 236/459] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:12:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:12:30.262517 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eabc89-f385-4f09-80ec-5aef58224842 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:30.335590 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e7626-7e09-4fdc-bc7f-7bd35af65121 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:30.336607 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e7626-7e09-4fdc-bc7f-7bd35af65121 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:30.337099 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e7626-7e09-4fdc-bc7f-7bd35af65121 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:30.339163 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 237/460] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:12:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:12:30.348194 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33083358-26d0-43da-9b66-6cc411e249d1 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:30.348870 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33083358-26d0-43da-9b66-6cc411e249d1 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:30.349392 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33083358-26d0-43da-9b66-6cc411e249d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:30.395600 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-33083358-26d0-43da-9b66-6cc411e249d1 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:12:30.397395 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-33083358-26d0-43da-9b66-6cc411e249d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:12:30.470542 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eabc89-f385-4f09-80ec-5aef58224842 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:30.529512 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be43cf3e-99f2-4570-b42e-900cfb466c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:30.531054 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be43cf3e-99f2-4570-b42e-900cfb466c89 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:30.531711 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be43cf3e-99f2-4570-b42e-900cfb466c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:30.532360 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be43cf3e-99f2-4570-b42e-900cfb466c89 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:30.533604 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be43cf3e-99f2-4570-b42e-900cfb466c89 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:30.536075 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be43cf3e-99f2-4570-b42e-900cfb466c89 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:30.568590 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:30.572705 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 224/461] 10.222.0.87 () {60 vars in 1132 bytes} [Tue Jun 21 18:12:30 2022] POST /identity/v3/users => generated 246 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:12:30.603233 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 238/462] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:12:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:12:30.661848 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be43cf3e-99f2-4570-b42e-900cfb466c89 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:30.727776 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb1185d-5ce4-4110-a5a5-fabb34ed26e9 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:30.728163 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb1185d-5ce4-4110-a5a5-fabb34ed26e9 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:30.728505 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb1185d-5ce4-4110-a5a5-fabb34ed26e9 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:30.731600 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 239/463] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:12:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:12:30.740226 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8039f8d5-4bd9-40f8-b490-3da8a1000dd1 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:30.740654 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8039f8d5-4bd9-40f8-b490-3da8a1000dd1 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:30.740959 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8039f8d5-4bd9-40f8-b490-3da8a1000dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:30.771144 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-8039f8d5-4bd9-40f8-b490-3da8a1000dd1 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:12:30.773344 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8039f8d5-4bd9-40f8-b490-3da8a1000dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:12:30.785284 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:30.914015 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be43cf3e-99f2-4570-b42e-900cfb466c89 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:30.924130 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 240/464] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:12:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:12:30.964446 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffbb535-b340-4247-a17f-744bf7a6e78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:30.967061 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffbb535-b340-4247-a17f-744bf7a6e78d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:30.967581 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffbb535-b340-4247-a17f-744bf7a6e78d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:30.967951 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffbb535-b340-4247-a17f-744bf7a6e78d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:30.968837 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8039f8d5-4bd9-40f8-b490-3da8a1000dd1 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:30.998044 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffbb535-b340-4247-a17f-744bf7a6e78d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:31.001499 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffbb535-b340-4247-a17f-744bf7a6e78d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:31.039520 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-bffbb535-b340-4247-a17f-744bf7a6e78d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:31.039520 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:31.039520 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:31.039520 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:31.039520 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:31.039520 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:31.039520 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:31.039520 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:31.039520 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:31.039520 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:31.039520 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:31.039520 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:31.039520 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:12:31.039520 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:12:31.039520 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:12:31.041544 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:12:31.041544 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:31.041544 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:31.041544 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:31.041544 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:31.041544 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:31.041544 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:31.041544 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:31.041544 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:31.041544 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:31.041544 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:31.041544 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:31.041544 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:31.041544 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:31.041544 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:31.041544 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:31.041544 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:31.043671 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:12:31.043671 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:12:31.043671 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:31.043671 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:12:31.045895 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 225/465] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:12:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:31.058758 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffbb535-b340-4247-a17f-744bf7a6e78d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:31.178625 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8039f8d5-4bd9-40f8-b490-3da8a1000dd1 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:31.213865 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:31.251510 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d1503-ea2e-412e-992d-a241d71e41a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:31.253139 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d1503-ea2e-412e-992d-a241d71e41a9 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:31.253848 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d1503-ea2e-412e-992d-a241d71e41a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:31.254512 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d1503-ea2e-412e-992d-a241d71e41a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:31.278608 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0d1503-ea2e-412e-992d-a241d71e41a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:31.281980 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0d1503-ea2e-412e-992d-a241d71e41a9 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:31.318870 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-5d0d1503-ea2e-412e-992d-a241d71e41a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:31.318870 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:31.318870 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:31.318870 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:31.318870 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:31.318870 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:31.318870 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:31.318870 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:31.318870 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:31.318870 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:31.318870 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:31.318870 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:31.318870 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:12:31.318870 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:12:31.318870 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:12:31.320822 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:12:31.320822 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:31.320822 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:31.320822 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:31.320822 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:31.320822 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:31.320822 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:31.320822 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:31.320822 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:31.320822 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:31.320822 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:31.320822 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:31.320822 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:31.320822 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:31.320822 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:31.320822 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:31.320822 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:31.322919 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:12:31.322919 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:12:31.322919 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:31.322919 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:12:31.326113 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 241/466] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:12:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:31.340069 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d1503-ea2e-412e-992d-a241d71e41a9 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:31.348570 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffbb535-b340-4247-a17f-744bf7a6e78d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:31.366055 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:31.449417 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e9efa9-0cc1-40d8-8c13-fd69f0412cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:31.449796 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e9efa9-0cc1-40d8-8c13-fd69f0412cba None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:31.450260 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e9efa9-0cc1-40d8-8c13-fd69f0412cba None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:31.450749 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e9efa9-0cc1-40d8-8c13-fd69f0412cba None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:31.452126 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e9efa9-0cc1-40d8-8c13-fd69f0412cba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:31.452653 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e9efa9-0cc1-40d8-8c13-fd69f0412cba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:31.455175 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e9efa9-0cc1-40d8-8c13-fd69f0412cba None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:31.477993 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 226/467] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:12:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 423 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:31.480045 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e0d3c-3259-4a88-ab73-f413a896df39 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:31.480521 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e0d3c-3259-4a88-ab73-f413a896df39 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:31.481091 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e0d3c-3259-4a88-ab73-f413a896df39 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:31.483569 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 227/468] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:12:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:12:31.485055 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e0d3c-3259-4a88-ab73-f413a896df39 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:31.541282 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d1503-ea2e-412e-992d-a241d71e41a9 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:31.610855 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a81ac5-4a97-4ff8-9cf9-abe51276ff39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:31.612186 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a81ac5-4a97-4ff8-9cf9-abe51276ff39 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:31.612726 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a81ac5-4a97-4ff8-9cf9-abe51276ff39 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:31.613564 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a81ac5-4a97-4ff8-9cf9-abe51276ff39 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:31.615004 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a81ac5-4a97-4ff8-9cf9-abe51276ff39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:31.615601 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a81ac5-4a97-4ff8-9cf9-abe51276ff39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:31.619360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a81ac5-4a97-4ff8-9cf9-abe51276ff39 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:31.652933 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 242/469] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:12:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:31.664080 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf362d5-05ab-4a47-a3cd-d0df090e3382 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:31.664692 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf362d5-05ab-4a47-a3cd-d0df090e3382 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:31.665550 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf362d5-05ab-4a47-a3cd-d0df090e3382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:31.701460 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e0d3c-3259-4a88-ab73-f413a896df39 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:31.702833 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-7cf362d5-05ab-4a47-a3cd-d0df090e3382 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:12:31.706586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7cf362d5-05ab-4a47-a3cd-d0df090e3382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:12:31.761699 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:31.776731 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3632d20-42df-4001-8b71-d6ea403490ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:31.779025 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3632d20-42df-4001-8b71-d6ea403490ec None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:31.780246 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3632d20-42df-4001-8b71-d6ea403490ec None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:31.781230 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3632d20-42df-4001-8b71-d6ea403490ec None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:31.783779 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3632d20-42df-4001-8b71-d6ea403490ec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:31.787900 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3632d20-42df-4001-8b71-d6ea403490ec None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:31.846634 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 228/470] 10.222.0.87 () {60 vars in 1132 bytes} [Tue Jun 21 18:12:31 2022] POST /identity/v3/users => generated 248 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:12:31.848332 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3632d20-42df-4001-8b71-d6ea403490ec None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:31.876131 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 243/471] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:12:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:12:31.939795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21439c11-8b86-457b-8663-9b77d6a8ce74 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:31.940840 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21439c11-8b86-457b-8663-9b77d6a8ce74 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:31.944362 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21439c11-8b86-457b-8663-9b77d6a8ce74 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:31.944665 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:31.944826 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 244/472] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:12:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:12:31.946091 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21439c11-8b86-457b-8663-9b77d6a8ce74 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:32.127299 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3632d20-42df-4001-8b71-d6ea403490ec None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:32.165158 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21439c11-8b86-457b-8663-9b77d6a8ce74 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:32.196792 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe96aa-76f8-4761-94fb-ab87327b444e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:32.198218 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe96aa-76f8-4761-94fb-ab87327b444e None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:32.198862 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe96aa-76f8-4761-94fb-ab87327b444e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:32.199483 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe96aa-76f8-4761-94fb-ab87327b444e None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:32.201982 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe96aa-76f8-4761-94fb-ab87327b444e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:32.209674 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe96aa-76f8-4761-94fb-ab87327b444e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:32.230595 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974d075f-0806-4312-983c-0cea45dfb83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:32.232277 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d075f-0806-4312-983c-0cea45dfb83f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:32.233224 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d075f-0806-4312-983c-0cea45dfb83f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:32.233432 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d075f-0806-4312-983c-0cea45dfb83f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:32.253114 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974d075f-0806-4312-983c-0cea45dfb83f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:32.253407 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 229/473] 10.222.0.87 () {60 vars in 1132 bytes} [Tue Jun 21 18:12:31 2022] POST /identity/v3/users => generated 248 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:12:32.254541 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc9653-76ff-49c0-ab01-2466c00d19a7 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:32.254834 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc9653-76ff-49c0-ab01-2466c00d19a7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:32.255119 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc9653-76ff-49c0-ab01-2466c00d19a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:32.256161 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974d075f-0806-4312-983c-0cea45dfb83f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:32.280537 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-82bc9653-76ff-49c0-ab01-2466c00d19a7 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:12:32.282193 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-82bc9653-76ff-49c0-ab01-2466c00d19a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:12:32.307475 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-974d075f-0806-4312-983c-0cea45dfb83f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:32.307475 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:32.307475 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:32.307475 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:32.307475 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:32.307475 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:32.307475 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:32.307475 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:32.307475 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:32.307475 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:32.307475 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:32.307475 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:32.307475 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:12:32.307475 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:12:32.307475 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:12:32.309197 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:12:32.309197 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:32.309197 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:32.309197 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:32.309197 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:32.309197 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:32.309197 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:32.309197 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:32.309197 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:32.309197 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:32.309197 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:32.309197 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:32.309197 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:32.309197 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:32.309197 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:32.309197 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:32.309197 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:32.310963 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:12:32.310963 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:12:32.310963 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:32.310963 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:12:32.310963 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 245/474] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:12:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:32.321386 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974d075f-0806-4312-983c-0cea45dfb83f None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:32.365245 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:32.454467 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 230/475] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:12:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:12:32.482771 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc9653-76ff-49c0-ab01-2466c00d19a7 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:32.488586 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:32.598421 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974d075f-0806-4312-983c-0cea45dfb83f None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:32.670382 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d65f4-7b60-4a88-b5c9-538e28ac62a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:32.671107 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d65f4-7b60-4a88-b5c9-538e28ac62a5 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:32.671495 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d65f4-7b60-4a88-b5c9-538e28ac62a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:32.671838 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d65f4-7b60-4a88-b5c9-538e28ac62a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:32.673238 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d65f4-7b60-4a88-b5c9-538e28ac62a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:32.673618 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d65f4-7b60-4a88-b5c9-538e28ac62a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:32.675415 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d65f4-7b60-4a88-b5c9-538e28ac62a5 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:32.685170 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc9653-76ff-49c0-ab01-2466c00d19a7 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:32.697979 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 246/476] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:12:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:32.707069 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d65f4-7b60-4a88-b5c9-538e28ac62a5 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:32.736370 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d280b646-a9c1-425b-9d2b-fddb26541be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:32.737062 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280b646-a9c1-425b-9d2b-fddb26541be8 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:32.737355 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280b646-a9c1-425b-9d2b-fddb26541be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:32.737658 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280b646-a9c1-425b-9d2b-fddb26541be8 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:32.739063 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d280b646-a9c1-425b-9d2b-fddb26541be8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:32.740022 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d280b646-a9c1-425b-9d2b-fddb26541be8 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:32.747518 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.sql.core [None req-d280b646-a9c1-425b-9d2b-fddb26541be8 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 21 18:12:32.747518 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 21 18:12:32.747518 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [parameters: {'id': '06c15b17e0c14cfe88ba25a8ebea6942', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 21 18:12:32.747518 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80399) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 21 18:12:32.768831 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-d280b646-a9c1-425b-9d2b-fddb26541be8 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 21 18:12:32.768831 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:32.768831 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 21 18:12:32.768831 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 21 18:12:32.768831 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 21 18:12:32.768831 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 21 18:12:32.768831 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 21 18:12:32.768831 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application result = self._query(query) Jun 21 18:12:32.768831 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 21 18:12:32.768831 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application conn.query(q) Jun 21 18:12:32.768831 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 21 18:12:32.768831 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 21 18:12:32.768831 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 21 18:12:32.768831 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application result.read() Jun 21 18:12:32.768831 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 21 18:12:32.770678 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 21 18:12:32.770678 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 21 18:12:32.770678 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 21 18:12:32.770678 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 21 18:12:32.770678 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 21 18:12:32.770678 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 21 18:12:32.770678 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 21 18:12:32.770678 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 21 18:12:32.770678 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:12:32.770678 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 21 18:12:32.770678 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:12:32.770678 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:32.770678 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 21 18:12:32.770678 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 21 18:12:32.770678 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 21 18:12:32.770678 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return ref.to_dict() Jun 21 18:12:32.770678 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 21 18:12:32.770678 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application next(self.gen) Jun 21 18:12:32.773281 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 21 18:12:32.773281 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application yield resource Jun 21 18:12:32.773281 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 21 18:12:32.773281 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application next(self.gen) Jun 21 18:12:32.773281 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 21 18:12:32.773281 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self.session.rollback() Jun 21 18:12:32.773281 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 21 18:12:32.773281 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self.force_reraise() Jun 21 18:12:32.773281 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 21 18:12:32.773281 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise self.value Jun 21 18:12:32.773281 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 21 18:12:32.773281 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 21 18:12:32.773281 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 21 18:12:32.773281 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application session.commit() Jun 21 18:12:32.773281 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 21 18:12:32.773281 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 21 18:12:32.773281 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 21 18:12:32.780813 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._prepare_impl() Jun 21 18:12:32.780813 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 21 18:12:32.780813 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self.session.flush() Jun 21 18:12:32.780813 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 21 18:12:32.780813 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._flush(objects) Jun 21 18:12:32.780813 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 21 18:12:32.780813 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 21 18:12:32.780813 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 21 18:12:32.780813 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application compat.raise_( Jun 21 18:12:32.780813 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 21 18:12:32.780813 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception Jun 21 18:12:32.780813 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 21 18:12:32.780813 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application flush_context.execute() Jun 21 18:12:32.780813 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 21 18:12:32.780813 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rec.execute(self) Jun 21 18:12:32.780813 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 21 18:12:32.780813 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 21 18:12:32.784264 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 21 18:12:32.784264 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 21 18:12:32.784264 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 21 18:12:32.784264 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 21 18:12:32.784264 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 21 18:12:32.784264 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 21 18:12:32.784264 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 21 18:12:32.784264 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 21 18:12:32.784264 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 21 18:12:32.784264 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 21 18:12:32.784264 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 21 18:12:32.784264 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 21 18:12:32.784264 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 21 18:12:32.784264 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 21 18:12:32.784264 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 21 18:12:32.784264 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception Jun 21 18:12:32.788619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 21 18:12:32.788619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 21 18:12:32.788619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 21 18:12:32.788619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 21 18:12:32.788619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 21 18:12:32.788619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application result = self._query(query) Jun 21 18:12:32.788619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 21 18:12:32.788619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application conn.query(q) Jun 21 18:12:32.788619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 21 18:12:32.788619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 21 18:12:32.788619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 21 18:12:32.788619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application result.read() Jun 21 18:12:32.788619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 21 18:12:32.788619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 21 18:12:32.788619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 21 18:12:32.788619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 21 18:12:32.796068 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 21 18:12:32.796068 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 21 18:12:32.796068 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 21 18:12:32.796068 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 21 18:12:32.796068 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 21 18:12:32.796068 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 21 18:12:32.796068 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [parameters: {'id': '06c15b17e0c14cfe88ba25a8ebea6942', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 21 18:12:32.796068 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 21 18:12:32.796068 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:12:32.796068 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:12:32.796068 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:12:32.796068 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:32.796068 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:32.796068 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:32.796068 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:32.796068 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:32.800241 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:32.800241 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:32.800241 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:32.800241 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:32.800241 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:32.800241 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:32.800241 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 21 18:12:32.800241 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 21 18:12:32.800241 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:32.800241 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:32.800241 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 21 18:12:32.800241 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 21 18:12:32.800241 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 21 18:12:32.800241 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 21 18:12:32.800241 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 21 18:12:32.800241 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:12:32.805243 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 231/477] 10.222.0.87 () {60 vars in 1131 bytes} [Tue Jun 21 18:12:32 2022] POST /identity/v3/roles => generated 144 bytes in 289 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:32.805243 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d280b646-a9c1-425b-9d2b-fddb26541be8 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:32.887209 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:32.916666 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:32.959866 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d65f4-7b60-4a88-b5c9-538e28ac62a5 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:32.963420 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d280b646-a9c1-425b-9d2b-fddb26541be8 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:33.006007 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d4cc9f-e334-49e2-87bc-550bafc99f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:33.006857 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4cc9f-e334-49e2-87bc-550bafc99f1d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:33.007142 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4cc9f-e334-49e2-87bc-550bafc99f1d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:33.007422 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4cc9f-e334-49e2-87bc-550bafc99f1d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:33.021287 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d4cc9f-e334-49e2-87bc-550bafc99f1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:12:33.037696 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-66d4cc9f-e334-49e2-87bc-550bafc99f1d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:12:33.037696 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:33.037696 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:33.037696 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:33.037696 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:33.037696 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:33.037696 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:33.037696 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:33.037696 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:33.037696 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:33.037696 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:33.037696 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:33.037696 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:12:33.037696 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:12:33.037696 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:12:33.037696 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:12:33.039493 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:12:33.039493 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:12:33.039493 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:12:33.039493 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:12:33.039493 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:33.039493 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:33.039493 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:33.039493 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:33.039493 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:33.039493 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:33.039493 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:33.039493 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:33.039493 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:33.039493 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:33.039493 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:33.039493 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:33.041120 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:33.041120 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:33.041120 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:33.041120 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:33.041120 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:12:33.041120 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:12:33.041120 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:12:33.041120 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:12:33.041120 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:12:33.041120 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:12:33.041120 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:12:33.041120 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:12:33.041120 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 232/478] 10.222.0.87 () {56 vars in 1114 bytes} [Tue Jun 21 18:12:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:33.044269 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d4cc9f-e334-49e2-87bc-550bafc99f1d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:33.052245 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e037672-331c-4cba-ab0c-ba9cfc9de077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:33.053472 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e037672-331c-4cba-ab0c-ba9cfc9de077 None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:33.054007 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e037672-331c-4cba-ab0c-ba9cfc9de077 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:33.054516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e037672-331c-4cba-ab0c-ba9cfc9de077 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:33.055918 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e037672-331c-4cba-ab0c-ba9cfc9de077 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:33.067581 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e037672-331c-4cba-ab0c-ba9cfc9de077 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:33.123223 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 247/479] 10.222.0.87 () {60 vars in 1132 bytes} [Tue Jun 21 18:12:32 2022] POST /identity/v3/users => generated 249 bytes in 417 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:12:33.288439 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d4cc9f-e334-49e2-87bc-550bafc99f1d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:33.358513 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:33.363380 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5060e094-f813-4978-8e85-277265bf8c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:33.364660 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060e094-f813-4978-8e85-277265bf8c66 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:33.365188 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060e094-f813-4978-8e85-277265bf8c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:33.365736 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060e094-f813-4978-8e85-277265bf8c66 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:33.367343 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5060e094-f813-4978-8e85-277265bf8c66 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:33.368179 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5060e094-f813-4978-8e85-277265bf8c66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:33.371494 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5060e094-f813-4978-8e85-277265bf8c66 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:33.389627 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 233/480] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:12:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:34.462582 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2f0e4-c6ee-4a55-9d4e-095fbe909481 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:34.463401 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2f0e4-c6ee-4a55-9d4e-095fbe909481 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:34.463544 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2f0e4-c6ee-4a55-9d4e-095fbe909481 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:34.467103 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 248/481] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:12:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:12:34.474116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223742b-22f6-4569-a1f0-993ff3063115 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:34.474948 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223742b-22f6-4569-a1f0-993ff3063115 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:34.475342 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223742b-22f6-4569-a1f0-993ff3063115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:34.508872 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-9223742b-22f6-4569-a1f0-993ff3063115 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:12:34.511838 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9223742b-22f6-4569-a1f0-993ff3063115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:12:34.663718 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 234/482] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:12:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:12:34.709146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f2f0e4-c6ee-4a55-9d4e-095fbe909481 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:35.025091 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f2f0e4-c6ee-4a55-9d4e-095fbe909481 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:35.090215 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:35.115001 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b719dff6-f319-4fc2-b50e-f77bcc0da17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:35.116395 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719dff6-f319-4fc2-b50e-f77bcc0da17f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:35.116889 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719dff6-f319-4fc2-b50e-f77bcc0da17f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:35.117390 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719dff6-f319-4fc2-b50e-f77bcc0da17f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:35.143499 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b719dff6-f319-4fc2-b50e-f77bcc0da17f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:12:35.157288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-b719dff6-f319-4fc2-b50e-f77bcc0da17f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:12:35.157288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:35.157288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:35.157288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:35.157288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:35.157288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:35.157288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:35.157288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:35.157288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:35.157288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:35.157288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:35.157288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:35.157288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:12:35.157288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:12:35.157288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:12:35.157288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:12:35.158363 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:12:35.158363 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:12:35.158363 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:12:35.158363 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:12:35.158363 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:35.158363 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:35.158363 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:35.158363 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:35.158363 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:35.158363 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:35.158363 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:35.158363 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:35.158363 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:35.158363 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:35.158363 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:35.158363 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:35.159220 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:35.159220 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:35.159220 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:35.159220 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:35.159220 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:12:35.159220 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:12:35.159220 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:12:35.159220 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:12:35.159220 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:12:35.159220 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:12:35.159220 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:12:35.159220 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:12:35.164802 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 249/483] 10.222.0.87 () {56 vars in 1102 bytes} [Tue Jun 21 18:12:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 457 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:35.175130 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9223742b-22f6-4569-a1f0-993ff3063115 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:35.365375 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf62138-40bb-436c-ae22-c2ce6805ea0a None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:35.365560 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf62138-40bb-436c-ae22-c2ce6805ea0a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:35.365859 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf62138-40bb-436c-ae22-c2ce6805ea0a None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:35.367345 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 250/484] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:12:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:12:35.377681 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd238b-b7ee-43c2-9b2d-a12005a79046 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:35.378308 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd238b-b7ee-43c2-9b2d-a12005a79046 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:35.379010 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd238b-b7ee-43c2-9b2d-a12005a79046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:35.421366 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-15bd238b-b7ee-43c2-9b2d-a12005a79046 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:12:35.422869 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-15bd238b-b7ee-43c2-9b2d-a12005a79046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:12:35.433908 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9223742b-22f6-4569-a1f0-993ff3063115 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:35.470205 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:35.533144 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5699d0-163b-4645-8efd-cc90c64f4f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:35.534295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5699d0-163b-4645-8efd-cc90c64f4f4d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:35.534778 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5699d0-163b-4645-8efd-cc90c64f4f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:35.535275 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5699d0-163b-4645-8efd-cc90c64f4f4d None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:35.536725 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5699d0-163b-4645-8efd-cc90c64f4f4d None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:35.539151 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5699d0-163b-4645-8efd-cc90c64f4f4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:35.541421 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5699d0-163b-4645-8efd-cc90c64f4f4d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:35.583528 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 235/485] 10.222.0.87 () {56 vars in 1103 bytes} [Tue Jun 21 18:12:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 410 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:35.613417 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5699d0-163b-4645-8efd-cc90c64f4f4d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:35.683423 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 251/486] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:12:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:12:35.727527 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bd238b-b7ee-43c2-9b2d-a12005a79046 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:35.916414 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bd238b-b7ee-43c2-9b2d-a12005a79046 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:35.920877 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5699d0-163b-4645-8efd-cc90c64f4f4d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:35.953042 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:36.001228 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4acb1b-dc11-4315-b3b7-760ad322c62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:36.002516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4acb1b-dc11-4315-b3b7-760ad322c62e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:36.003014 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4acb1b-dc11-4315-b3b7-760ad322c62e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:36.003544 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4acb1b-dc11-4315-b3b7-760ad322c62e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:36.026014 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de31293e-489f-42e6-8f5d-f7a422de1219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:36.027396 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31293e-489f-42e6-8f5d-f7a422de1219 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:36.027884 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31293e-489f-42e6-8f5d-f7a422de1219 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:36.028372 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31293e-489f-42e6-8f5d-f7a422de1219 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:36.052825 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4acb1b-dc11-4315-b3b7-760ad322c62e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:12:36.064084 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-5e4acb1b-dc11-4315-b3b7-760ad322c62e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:12:36.064084 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:36.064084 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:36.064084 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:36.064084 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:36.064084 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:36.064084 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:36.064084 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:36.064084 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:36.064084 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:36.064084 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:36.064084 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:36.064084 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:12:36.064084 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:12:36.064084 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:12:36.064084 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:12:36.066167 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:12:36.066167 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:12:36.066167 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:12:36.066167 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:12:36.066167 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:36.066167 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:36.066167 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:36.066167 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:36.066167 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:36.066167 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:36.066167 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:36.066167 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:36.066167 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:36.066167 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:36.066167 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:36.066167 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:36.067650 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:36.067650 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:36.067650 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:36.067650 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:36.067650 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:12:36.067650 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:12:36.067650 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:12:36.067650 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:12:36.067650 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:12:36.067650 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:12:36.067650 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:12:36.067650 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:12:36.067650 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 252/487] 10.222.0.87 () {56 vars in 1102 bytes} [Tue Jun 21 18:12:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:12:36.080231 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de31293e-489f-42e6-8f5d-f7a422de1219 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:36.082762 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de31293e-489f-42e6-8f5d-f7a422de1219 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:36.084919 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4acb1b-dc11-4315-b3b7-760ad322c62e None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:36.124590 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-de31293e-489f-42e6-8f5d-f7a422de1219 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:36.124590 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:36.124590 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:36.124590 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:36.124590 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:36.124590 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:36.124590 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:36.124590 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:36.124590 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:36.124590 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:36.124590 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:36.124590 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:36.124590 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:12:36.124590 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:12:36.124590 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:12:36.127478 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:12:36.127478 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:36.127478 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:36.127478 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:36.127478 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:36.127478 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:36.127478 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:36.127478 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:36.127478 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:36.127478 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:36.127478 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:36.127478 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:36.127478 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:36.127478 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:36.127478 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:36.127478 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:36.127478 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:36.130837 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:12:36.130837 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:12:36.130837 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:36.130837 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:12:36.131492 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 236/488] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:12:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 514 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:12:36.135129 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de31293e-489f-42e6-8f5d-f7a422de1219 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:36.274890 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:36.280745 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4acb1b-dc11-4315-b3b7-760ad322c62e None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:36.366947 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ef6d4-428f-4a3b-8467-1875b1ab168a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:36.368064 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ef6d4-428f-4a3b-8467-1875b1ab168a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:36.368536 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ef6d4-428f-4a3b-8467-1875b1ab168a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:36.369001 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ef6d4-428f-4a3b-8467-1875b1ab168a None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:36.370482 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5ef6d4-428f-4a3b-8467-1875b1ab168a None admin] RBAC: Adding query filter params (name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:36.371007 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5ef6d4-428f-4a3b-8467-1875b1ab168a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:36.373671 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5ef6d4-428f-4a3b-8467-1875b1ab168a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:36.390534 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 253/489] 10.222.0.87 () {56 vars in 1103 bytes} [Tue Jun 21 18:12:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:36.403440 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ef6d4-428f-4a3b-8467-1875b1ab168a None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:36.494081 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de31293e-489f-42e6-8f5d-f7a422de1219 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:36.540462 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:36.595055 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c98ea6c-8ad4-42cc-a49b-4334144968e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:36.596199 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c98ea6c-8ad4-42cc-a49b-4334144968e8 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:36.599354 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c98ea6c-8ad4-42cc-a49b-4334144968e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:36.599856 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c98ea6c-8ad4-42cc-a49b-4334144968e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:36.601560 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c98ea6c-8ad4-42cc-a49b-4334144968e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:36.602066 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c98ea6c-8ad4-42cc-a49b-4334144968e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:36.604663 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c98ea6c-8ad4-42cc-a49b-4334144968e8 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:36.650349 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 237/490] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:12:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 517 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:36.675828 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c98ea6c-8ad4-42cc-a49b-4334144968e8 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:36.692734 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ef6d4-428f-4a3b-8467-1875b1ab168a None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:36.750879 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec34954-9dd3-463d-8380-f4704c85a298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:36.752382 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec34954-9dd3-463d-8380-f4704c85a298 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:36.752869 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec34954-9dd3-463d-8380-f4704c85a298 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:36.753406 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec34954-9dd3-463d-8380-f4704c85a298 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:36.774607 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec34954-9dd3-463d-8380-f4704c85a298 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:36.777122 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec34954-9dd3-463d-8380-f4704c85a298 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:36.814634 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-fec34954-9dd3-463d-8380-f4704c85a298 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:36.814634 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:36.814634 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:36.814634 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:36.814634 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:36.814634 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:36.814634 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:36.814634 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:36.814634 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:36.814634 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:36.814634 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:36.814634 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:36.814634 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:12:36.814634 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:12:36.814634 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:12:36.816955 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:12:36.816955 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:36.816955 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:36.816955 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:36.816955 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:36.816955 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:36.816955 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:36.816955 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:36.816955 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:36.816955 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:36.816955 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:36.816955 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:36.816955 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:36.816955 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:36.816955 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:36.816955 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:36.816955 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:36.818736 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:12:36.818736 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:12:36.818736 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:36.818736 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:12:36.818736 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 254/491] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:12:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 412 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:36.818736 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d2a254-5cac-4251-9461-71506f776df3 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:36.818736 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d2a254-5cac-4251-9461-71506f776df3 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:36.818736 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d2a254-5cac-4251-9461-71506f776df3 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:36.827095 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 255/492] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:12:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:12:36.827095 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d2a254-5cac-4251-9461-71506f776df3 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:36.901958 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:36.919174 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c98ea6c-8ad4-42cc-a49b-4334144968e8 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:36.992309 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c1443a-16f4-4d76-b72d-7b774db86bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:36.993763 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1443a-16f4-4d76-b72d-7b774db86bf1 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:36.994282 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1443a-16f4-4d76-b72d-7b774db86bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:36.994770 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1443a-16f4-4d76-b72d-7b774db86bf1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:37.011450 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c1443a-16f4-4d76-b72d-7b774db86bf1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:37.015452 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c1443a-16f4-4d76-b72d-7b774db86bf1 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:37.025765 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:37.047968 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d2a254-5cac-4251-9461-71506f776df3 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:37.049141 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-c1c1443a-16f4-4d76-b72d-7b774db86bf1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:12:37.049141 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:37.049141 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:37.049141 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:37.049141 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:37.049141 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:37.049141 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:37.049141 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:37.049141 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:37.049141 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:37.049141 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:37.049141 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:37.049141 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:12:37.049141 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:12:37.049141 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:12:37.049141 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:12:37.050758 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:37.050758 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:37.050758 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:12:37.050758 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:12:37.050758 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:12:37.050758 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:12:37.050758 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:37.050758 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:37.050758 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:37.050758 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:37.050758 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:37.050758 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:37.050758 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:37.050758 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:37.050758 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:37.050758 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:37.050758 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:37.050758 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:37.054404 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:37.054404 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:37.054404 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:12:37.054404 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:12:37.054404 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:12:37.054404 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:12:37.054404 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:12:37.054404 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:12:37.054404 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:12:37.054404 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:12:37.054404 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 238/493] 10.222.0.87 () {56 vars in 1093 bytes} [Tue Jun 21 18:12:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:12:37.054404 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e7bd4-b602-46ac-a362-a3db1a1256bd None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:37.055057 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e7bd4-b602-46ac-a362-a3db1a1256bd None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:37.055356 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e7bd4-b602-46ac-a362-a3db1a1256bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:37.093506 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a66e7bd4-b602-46ac-a362-a3db1a1256bd None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:12:37.095930 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a66e7bd4-b602-46ac-a362-a3db1a1256bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:12:37.129858 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3637463a-a777-47af-8a9a-aeaa6949b079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:37.131017 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637463a-a777-47af-8a9a-aeaa6949b079 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:37.131501 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637463a-a777-47af-8a9a-aeaa6949b079 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:37.131985 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637463a-a777-47af-8a9a-aeaa6949b079 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:37.133564 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3637463a-a777-47af-8a9a-aeaa6949b079 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:37.134288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3637463a-a777-47af-8a9a-aeaa6949b079 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:37.136896 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3637463a-a777-47af-8a9a-aeaa6949b079 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:37.162726 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 256/494] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:12:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:37.164524 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3637463a-a777-47af-8a9a-aeaa6949b079 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:37.237269 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 239/495] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:12:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:12:37.238523 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66e7bd4-b602-46ac-a362-a3db1a1256bd None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:37.343887 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3637463a-a777-47af-8a9a-aeaa6949b079 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:37.348669 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:37.371634 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66e7bd4-b602-46ac-a362-a3db1a1256bd None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:37.396186 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4601f965-2b75-4c36-87a5-15da98d36c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:37.397786 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4601f965-2b75-4c36-87a5-15da98d36c2e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:37.398432 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4601f965-2b75-4c36-87a5-15da98d36c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:37.398751 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4601f965-2b75-4c36-87a5-15da98d36c2e None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:37.400383 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4601f965-2b75-4c36-87a5-15da98d36c2e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:37.400929 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4601f965-2b75-4c36-87a5-15da98d36c2e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:37.404241 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4601f965-2b75-4c36-87a5-15da98d36c2e None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:37.413971 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e607c0-fbd8-4084-802f-cf8d3a6cd5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:37.414939 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e607c0-fbd8-4084-802f-cf8d3a6cd5f6 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:37.415346 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e607c0-fbd8-4084-802f-cf8d3a6cd5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:37.415733 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e607c0-fbd8-4084-802f-cf8d3a6cd5f6 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:37.424583 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e607c0-fbd8-4084-802f-cf8d3a6cd5f6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:37.426619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e607c0-fbd8-4084-802f-cf8d3a6cd5f6 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:37.446131 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-05e607c0-fbd8-4084-802f-cf8d3a6cd5f6 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 21 18:12:37.446131 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:37.446131 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:37.446131 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:37.446131 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:37.446131 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:37.446131 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:37.446131 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:37.446131 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:37.446131 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:37.446131 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:37.446131 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:37.446131 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:12:37.446131 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:12:37.446131 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:12:37.446131 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:12:37.447404 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:37.447404 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:37.447404 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:12:37.447404 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:12:37.447404 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:12:37.447404 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:12:37.447404 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:37.447404 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:37.447404 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:37.447404 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:37.447404 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:37.447404 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:37.447404 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:37.447404 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:37.447404 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:37.447404 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:37.447404 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:37.447404 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:37.448315 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:37.448315 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:37.448315 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:12:37.448315 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:12:37.448315 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:12:37.448315 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:12:37.448315 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:12:37.448315 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:12:37.448315 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 21 18:12:37.448315 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:12:37.448315 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 240/496] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:12:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:37.448887 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3a698-3b5f-4f4c-9a13-0af86b2d0c56 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:37.449323 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3a698-3b5f-4f4c-9a13-0af86b2d0c56 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:37.449323 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3a698-3b5f-4f4c-9a13-0af86b2d0c56 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:37.450910 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 241/497] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:12:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:12:37.453323 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f492bd-ecbf-438a-9713-b8a73e40de6a None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:37.454181 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f492bd-ecbf-438a-9713-b8a73e40de6a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:37.454678 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f492bd-ecbf-438a-9713-b8a73e40de6a None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:37.457080 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 242/498] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:12:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:12:37.459002 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f492bd-ecbf-438a-9713-b8a73e40de6a None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:37.474873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 257/499] 10.222.0.87 () {56 vars in 1133 bytes} [Tue Jun 21 18:12:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:37.476387 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aaf7b0-0810-446a-9f35-e381e63af88a None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:37.476769 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aaf7b0-0810-446a-9f35-e381e63af88a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:37.476932 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aaf7b0-0810-446a-9f35-e381e63af88a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:37.509020 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-84aaf7b0-0810-446a-9f35-e381e63af88a None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:12:37.511261 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-84aaf7b0-0810-446a-9f35-e381e63af88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:12:37.517569 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:37.610573 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 258/500] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:12:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:12:37.612450 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aaf7b0-0810-446a-9f35-e381e63af88a None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:37.631582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f492bd-ecbf-438a-9713-b8a73e40de6a None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:37.679262 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff617d3-6286-49e4-a738-adc6c68cb4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:37.680096 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff617d3-6286-49e4-a738-adc6c68cb4df None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:37.680384 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff617d3-6286-49e4-a738-adc6c68cb4df None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:37.680665 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff617d3-6286-49e4-a738-adc6c68cb4df None admin] PATH_INFO: `/v3/roles/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:37.696371 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff617d3-6286-49e4-a738-adc6c68cb4df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:12:37.712175 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-bff617d3-6286-49e4-a738-adc6c68cb4df None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:12:37.712175 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:37.712175 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:37.712175 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:37.712175 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:37.712175 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:37.712175 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:37.712175 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:37.712175 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:37.712175 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:37.712175 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:37.712175 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:37.712175 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:12:37.712175 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:12:37.712175 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:12:37.712175 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:12:37.713949 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:12:37.713949 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:12:37.713949 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:12:37.713949 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:12:37.713949 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:37.713949 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:37.713949 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:37.713949 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:37.713949 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:37.713949 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:37.713949 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:37.713949 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:37.713949 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:37.713949 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:37.713949 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:37.713949 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:37.715454 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:37.715454 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:37.715454 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:37.715454 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:37.715454 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:12:37.715454 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:12:37.715454 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:12:37.715454 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:12:37.715454 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:12:37.715454 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:12:37.715454 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:12:37.715454 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:12:37.715454 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 243/501] 10.222.0.87 () {56 vars in 1102 bytes} [Tue Jun 21 18:12:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:37.717166 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5e934-905a-433c-8c9b-03ce9e0a6e7e None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:37.717426 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5e934-905a-433c-8c9b-03ce9e0a6e7e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:37.718341 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5e934-905a-433c-8c9b-03ce9e0a6e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:37.729207 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:37.747906 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-18f5e934-905a-433c-8c9b-03ce9e0a6e7e None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:12:37.749454 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-18f5e934-905a-433c-8c9b-03ce9e0a6e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:12:37.784957 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aaf7b0-0810-446a-9f35-e381e63af88a None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:37.847041 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc110715-5fd3-472e-92f1-c68cfe2e4476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:37.848259 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc110715-5fd3-472e-92f1-c68cfe2e4476 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:37.848891 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc110715-5fd3-472e-92f1-c68cfe2e4476 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:37.849229 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc110715-5fd3-472e-92f1-c68cfe2e4476 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:37.850878 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc110715-5fd3-472e-92f1-c68cfe2e4476 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:37.851424 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc110715-5fd3-472e-92f1-c68cfe2e4476 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:37.854894 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc110715-5fd3-472e-92f1-c68cfe2e4476 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:37.862312 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 244/502] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:12:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:12:37.863282 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f5e934-905a-433c-8c9b-03ce9e0a6e7e None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:37.874325 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:37.904018 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 259/503] 10.222.0.87 () {56 vars in 1137 bytes} [Tue Jun 21 18:12:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:37.905411 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc110715-5fd3-472e-92f1-c68cfe2e4476 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:38.037766 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f5e934-905a-433c-8c9b-03ce9e0a6e7e None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:38.045476 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc110715-5fd3-472e-92f1-c68cfe2e4476 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:38.100275 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1855974-d561-4af4-962c-eed5a345ee0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:38.101119 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1855974-d561-4af4-962c-eed5a345ee0b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:38.101403 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1855974-d561-4af4-962c-eed5a345ee0b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:38.101732 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1855974-d561-4af4-962c-eed5a345ee0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:38.109677 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741d20d1-e9e4-4360-bb2f-d74b4df27db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:38.110434 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d20d1-e9e4-4360-bb2f-d74b4df27db8 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:38.110740 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d20d1-e9e4-4360-bb2f-d74b4df27db8 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:38.111060 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d20d1-e9e4-4360-bb2f-d74b4df27db8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:38.114908 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:38.125223 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1855974-d561-4af4-962c-eed5a345ee0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:38.126998 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1855974-d561-4af4-962c-eed5a345ee0b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:38.158063 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-c1855974-d561-4af4-962c-eed5a345ee0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:38.158063 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:38.158063 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:38.158063 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:38.158063 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:38.158063 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:38.158063 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:38.158063 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:38.158063 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:38.158063 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:38.158063 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:38.158063 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:38.158063 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:12:38.158063 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:12:38.158063 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:12:38.160333 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:12:38.160333 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:38.160333 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:38.160333 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:38.160333 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:38.160333 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:38.160333 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:38.160333 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:38.160333 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:38.160333 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:38.160333 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:38.160333 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:38.160333 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:38.160333 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:38.160333 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:38.160333 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:38.160333 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:38.162046 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:12:38.162046 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:12:38.162046 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:38.162046 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:12:38.162046 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 260/504] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:12:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:38.162046 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1855974-d561-4af4-962c-eed5a345ee0b None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:38.240756 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:38.245539 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741d20d1-e9e4-4360-bb2f-d74b4df27db8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:38.248472 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741d20d1-e9e4-4360-bb2f-d74b4df27db8 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:38.330317 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1855974-d561-4af4-962c-eed5a345ee0b None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:38.381120 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb2d1f-1f17-4c0d-b955-9b3a6e12fdbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:38.381767 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb2d1f-1f17-4c0d-b955-9b3a6e12fdbe None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:38.382090 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb2d1f-1f17-4c0d-b955-9b3a6e12fdbe None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:38.382380 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb2d1f-1f17-4c0d-b955-9b3a6e12fdbe None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:38.383377 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fb2d1f-1f17-4c0d-b955-9b3a6e12fdbe None admin] RBAC: Adding query filter params (name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:38.383695 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fb2d1f-1f17-4c0d-b955-9b3a6e12fdbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:38.385312 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fb2d1f-1f17-4c0d-b955-9b3a6e12fdbe None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:38.396753 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 261/505] 10.222.0.87 () {56 vars in 1103 bytes} [Tue Jun 21 18:12:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:38.397934 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb2d1f-1f17-4c0d-b955-9b3a6e12fdbe None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:38.466836 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 245/506] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:12:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:12:38.468038 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741d20d1-e9e4-4360-bb2f-d74b4df27db8 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:38.532331 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:38.577583 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb2d1f-1f17-4c0d-b955-9b3a6e12fdbe None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:38.600402 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:38.641785 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d185f-4af3-42b4-b003-8da1738f2cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:38.642831 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d185f-4af3-42b4-b003-8da1738f2cce None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:38.643173 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d185f-4af3-42b4-b003-8da1738f2cce None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:38.643607 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d185f-4af3-42b4-b003-8da1738f2cce None admin] PATH_INFO: `/v3/roles/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:38.660055 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741d20d1-e9e4-4360-bb2f-d74b4df27db8 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:38.661782 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813d185f-4af3-42b4-b003-8da1738f2cce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:12:38.671611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-813d185f-4af3-42b4-b003-8da1738f2cce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:12:38.671611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:38.671611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:38.671611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:38.671611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:38.671611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:38.671611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:38.671611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:38.671611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:38.671611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:38.671611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:38.671611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:38.671611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:12:38.671611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:12:38.671611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:12:38.671611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:12:38.673621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:12:38.673621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:12:38.673621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:12:38.673621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:12:38.673621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:38.673621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:38.673621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:38.673621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:38.673621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:38.673621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:38.673621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:38.673621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:38.673621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:38.673621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:38.673621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:38.673621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:38.676745 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:38.676745 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:38.676745 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:38.676745 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:38.676745 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:12:38.676745 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:12:38.676745 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:12:38.676745 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:12:38.676745 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:12:38.676745 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:12:38.676745 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:12:38.676745 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:12:38.676745 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 262/507] 10.222.0.87 () {56 vars in 1102 bytes} [Tue Jun 21 18:12:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:38.676745 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d185f-4af3-42b4-b003-8da1738f2cce None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:38.713563 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139b5748-7e50-4779-91b7-9fe544f9db7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:38.714512 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b5748-7e50-4779-91b7-9fe544f9db7c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:38.714843 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b5748-7e50-4779-91b7-9fe544f9db7c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:38.715094 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b5748-7e50-4779-91b7-9fe544f9db7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:38.844005 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139b5748-7e50-4779-91b7-9fe544f9db7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:38.846489 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139b5748-7e50-4779-91b7-9fe544f9db7c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:38.858631 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d185f-4af3-42b4-b003-8da1738f2cce None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:38.905963 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:38.921047 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297f7a7e-f2ed-421b-afe0-d99161a18e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:38.922146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f7a7e-f2ed-421b-afe0-d99161a18e9e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:38.922515 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f7a7e-f2ed-421b-afe0-d99161a18e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:38.922839 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f7a7e-f2ed-421b-afe0-d99161a18e9e None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:38.923814 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297f7a7e-f2ed-421b-afe0-d99161a18e9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:38.924255 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297f7a7e-f2ed-421b-afe0-d99161a18e9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:38.926162 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297f7a7e-f2ed-421b-afe0-d99161a18e9e None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:38.949968 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 263/508] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:12:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:38.949968 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297f7a7e-f2ed-421b-afe0-d99161a18e9e None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:39.010349 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:39.032445 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 246/509] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:12:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:12:39.034270 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139b5748-7e50-4779-91b7-9fe544f9db7c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:39.139885 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297f7a7e-f2ed-421b-afe0-d99161a18e9e None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:39.196102 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc7eba-c69b-45f4-8289-17c6661d7733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:39.197542 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc7eba-c69b-45f4-8289-17c6661d7733 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:39.198198 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc7eba-c69b-45f4-8289-17c6661d7733 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:39.198660 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc7eba-c69b-45f4-8289-17c6661d7733 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:39.220894 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dc7eba-c69b-45f4-8289-17c6661d7733 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:39.222958 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dc7eba-c69b-45f4-8289-17c6661d7733 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:39.237674 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139b5748-7e50-4779-91b7-9fe544f9db7c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:39.250967 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-40dc7eba-c69b-45f4-8289-17c6661d7733 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:39.250967 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:39.250967 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:39.250967 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:39.250967 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:39.250967 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:39.250967 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:39.250967 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:39.250967 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:39.250967 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:39.250967 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:39.250967 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:39.250967 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:12:39.250967 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:12:39.250967 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:12:39.252493 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:12:39.252493 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:39.252493 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:39.252493 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:39.252493 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:39.252493 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:39.252493 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:39.252493 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:39.252493 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:39.252493 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:39.252493 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:39.252493 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:39.252493 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:39.252493 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:39.252493 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:39.252493 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:39.252493 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:39.253973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:12:39.253973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:12:39.253973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:39.253973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:12:39.253973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 264/510] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:12:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:39.254460 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc7eba-c69b-45f4-8289-17c6661d7733 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:39.301310 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3063fabe-42aa-4bad-9164-61bed88bb550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:39.302291 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3063fabe-42aa-4bad-9164-61bed88bb550 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:39.302634 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3063fabe-42aa-4bad-9164-61bed88bb550 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:39.302943 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3063fabe-42aa-4bad-9164-61bed88bb550 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:39.306998 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:39.326681 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3063fabe-42aa-4bad-9164-61bed88bb550 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:39.329396 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3063fabe-42aa-4bad-9164-61bed88bb550 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:39.365887 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-3063fabe-42aa-4bad-9164-61bed88bb550 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:39.365887 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:39.365887 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:39.365887 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:39.365887 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:39.365887 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:39.365887 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:39.365887 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:39.365887 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:39.365887 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:39.365887 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:39.365887 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:39.365887 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:12:39.365887 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:12:39.365887 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:12:39.367602 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:12:39.367602 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:39.367602 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:39.367602 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:39.367602 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:39.367602 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:39.367602 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:39.367602 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:39.367602 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:39.367602 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:39.367602 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:39.367602 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:39.367602 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:39.367602 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:39.367602 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:39.367602 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:39.367602 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:39.370529 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:12:39.370529 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:12:39.370529 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:39.370529 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:12:39.370529 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 247/511] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:12:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:39.372466 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3063fabe-42aa-4bad-9164-61bed88bb550 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:39.404492 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:39.434126 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc7eba-c69b-45f4-8289-17c6661d7733 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:39.495418 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54a8b26-97ff-4e6c-9d44-dbdc2eca600c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:39.496411 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54a8b26-97ff-4e6c-9d44-dbdc2eca600c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:39.496859 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54a8b26-97ff-4e6c-9d44-dbdc2eca600c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:39.497580 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54a8b26-97ff-4e6c-9d44-dbdc2eca600c None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:39.499175 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54a8b26-97ff-4e6c-9d44-dbdc2eca600c None admin] RBAC: Adding query filter params (name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:39.499726 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54a8b26-97ff-4e6c-9d44-dbdc2eca600c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:39.502672 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54a8b26-97ff-4e6c-9d44-dbdc2eca600c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:39.519376 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 265/512] 10.222.0.87 () {56 vars in 1103 bytes} [Tue Jun 21 18:12:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:39.520972 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54a8b26-97ff-4e6c-9d44-dbdc2eca600c None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:39.557890 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3063fabe-42aa-4bad-9164-61bed88bb550 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:39.603912 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb81d4f-b8fb-46d6-b024-0084b2b6263c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:39.604637 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb81d4f-b8fb-46d6-b024-0084b2b6263c None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:39.604926 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb81d4f-b8fb-46d6-b024-0084b2b6263c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:39.605213 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb81d4f-b8fb-46d6-b024-0084b2b6263c None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:39.605987 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb81d4f-b8fb-46d6-b024-0084b2b6263c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:39.607583 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb81d4f-b8fb-46d6-b024-0084b2b6263c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:39.635442 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 248/513] 10.222.0.87 () {60 vars in 1132 bytes} [Tue Jun 21 18:12:39 2022] POST /identity/v3/users => generated 247 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:12:39.637161 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb81d4f-b8fb-46d6-b024-0084b2b6263c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:39.676639 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54a8b26-97ff-4e6c-9d44-dbdc2eca600c None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:39.739833 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b38840-749b-4764-9997-dfba6399b4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:39.740907 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b38840-749b-4764-9997-dfba6399b4dd None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:39.741427 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b38840-749b-4764-9997-dfba6399b4dd None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:39.741979 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b38840-749b-4764-9997-dfba6399b4dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:39.743715 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:39.762590 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b38840-749b-4764-9997-dfba6399b4dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:39.768496 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b38840-749b-4764-9997-dfba6399b4dd None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:39.777390 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:39.800626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-c3b38840-749b-4764-9997-dfba6399b4dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:39.800626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:39.800626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:39.800626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:39.800626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:39.800626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:39.800626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:39.800626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:39.800626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:39.800626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:39.800626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:39.800626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:39.800626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:12:39.800626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:12:39.800626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:12:39.801986 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:12:39.801986 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:39.801986 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:39.801986 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:39.801986 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:39.801986 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:39.801986 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:39.801986 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:39.801986 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:39.801986 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:39.801986 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:39.801986 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:39.801986 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:39.801986 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:39.801986 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:39.801986 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:39.801986 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:39.803044 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:12:39.803044 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:12:39.803044 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:39.803044 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:12:39.803044 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 266/514] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:12:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:39.804575 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b38840-749b-4764-9997-dfba6399b4dd None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:39.886919 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb81d4f-b8fb-46d6-b024-0084b2b6263c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:39.972655 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6e3fa-b90c-4061-bb00-9ce49034f771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:39.973991 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6e3fa-b90c-4061-bb00-9ce49034f771 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:39.974494 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6e3fa-b90c-4061-bb00-9ce49034f771 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:39.974996 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6e3fa-b90c-4061-bb00-9ce49034f771 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:39.976469 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b6e3fa-b90c-4061-bb00-9ce49034f771 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:39.977139 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b6e3fa-b90c-4061-bb00-9ce49034f771 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:39.980218 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b6e3fa-b90c-4061-bb00-9ce49034f771 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:40.009617 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 249/515] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:12:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:40.011716 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6e3fa-b90c-4061-bb00-9ce49034f771 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:40.039519 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b38840-749b-4764-9997-dfba6399b4dd None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:40.116196 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8c07bb-c360-4450-8f55-829d8358fea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:40.117350 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c07bb-c360-4450-8f55-829d8358fea4 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:40.117873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c07bb-c360-4450-8f55-829d8358fea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:40.118363 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c07bb-c360-4450-8f55-829d8358fea4 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:40.119766 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8c07bb-c360-4450-8f55-829d8358fea4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:40.120311 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8c07bb-c360-4450-8f55-829d8358fea4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:40.123578 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8c07bb-c360-4450-8f55-829d8358fea4 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:40.162744 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 267/516] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:12:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:40.162744 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8c07bb-c360-4450-8f55-829d8358fea4 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:40.233493 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6e3fa-b90c-4061-bb00-9ce49034f771 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:40.235433 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:40.287015 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:40.328950 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abd2393-62d1-4935-b4c9-1129d85fbdb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:40.330286 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abd2393-62d1-4935-b4c9-1129d85fbdb9 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:40.331124 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abd2393-62d1-4935-b4c9-1129d85fbdb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:40.331323 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abd2393-62d1-4935-b4c9-1129d85fbdb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:40.352560 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abd2393-62d1-4935-b4c9-1129d85fbdb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:40.354842 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abd2393-62d1-4935-b4c9-1129d85fbdb9 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:40.379630 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8c07bb-c360-4450-8f55-829d8358fea4 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:40.391973 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-7abd2393-62d1-4935-b4c9-1129d85fbdb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:40.391973 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:40.391973 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:40.391973 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:40.391973 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:40.391973 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:40.391973 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:40.391973 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:40.391973 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:40.391973 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:40.391973 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:40.391973 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:40.391973 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:12:40.391973 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:12:40.391973 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:12:40.393023 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:12:40.393023 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:40.393023 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:40.393023 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:40.393023 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:40.393023 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:40.393023 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:40.393023 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:40.393023 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:40.393023 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:40.393023 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:40.393023 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:40.393023 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:40.393023 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:40.393023 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:40.393023 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:40.393023 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:40.394409 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:12:40.394409 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:12:40.394409 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:40.394409 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:12:40.401530 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 250/517] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:12:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 391 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:40.403177 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abd2393-62d1-4935-b4c9-1129d85fbdb9 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:40.455595 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8176ae-6d84-49af-b92b-2c48e36e22c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:40.456139 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8176ae-6d84-49af-b92b-2c48e36e22c3 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:40.456635 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8176ae-6d84-49af-b92b-2c48e36e22c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:40.457121 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8176ae-6d84-49af-b92b-2c48e36e22c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:40.459809 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8176ae-6d84-49af-b92b-2c48e36e22c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:40.459809 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8176ae-6d84-49af-b92b-2c48e36e22c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:40.462311 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8176ae-6d84-49af-b92b-2c48e36e22c3 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:40.483218 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 268/518] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:12:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:40.484270 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8176ae-6d84-49af-b92b-2c48e36e22c3 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:40.631753 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abd2393-62d1-4935-b4c9-1129d85fbdb9 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:40.712684 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9ad7e4-eb6f-45dd-a69f-70357deec90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:40.713849 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ad7e4-eb6f-45dd-a69f-70357deec90d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:40.714298 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ad7e4-eb6f-45dd-a69f-70357deec90d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:40.714826 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ad7e4-eb6f-45dd-a69f-70357deec90d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:40.734383 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9ad7e4-eb6f-45dd-a69f-70357deec90d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:40.736301 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9ad7e4-eb6f-45dd-a69f-70357deec90d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:40.738572 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8176ae-6d84-49af-b92b-2c48e36e22c3 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:40.761920 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-7f9ad7e4-eb6f-45dd-a69f-70357deec90d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 21 18:12:40.761920 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:40.761920 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:40.761920 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:40.761920 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:40.761920 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:40.761920 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:40.761920 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:40.761920 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:40.761920 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:40.761920 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:40.761920 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:40.761920 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:12:40.761920 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:12:40.761920 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:12:40.761920 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:12:40.764849 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:40.764849 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:40.764849 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:12:40.764849 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:12:40.764849 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:12:40.764849 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:12:40.764849 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:40.764849 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:40.764849 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:40.764849 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:40.764849 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:40.764849 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:40.764849 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:40.764849 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:40.764849 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:40.764849 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:40.764849 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:40.764849 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:40.768460 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:40.768460 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:40.768460 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:12:40.768460 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:12:40.768460 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:12:40.768460 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:12:40.768460 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:12:40.768460 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:12:40.768460 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 21 18:12:40.768460 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:12:40.768460 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 251/519] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:12:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:40.768460 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9ad7e4-eb6f-45dd-a69f-70357deec90d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:40.784022 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:40.806473 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:40.835737 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6904bb28-9deb-4ac4-9e0d-64c2e0436e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:40.836973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6904bb28-9deb-4ac4-9e0d-64c2e0436e08 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:40.837384 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6904bb28-9deb-4ac4-9e0d-64c2e0436e08 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:40.837875 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6904bb28-9deb-4ac4-9e0d-64c2e0436e08 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:40.853876 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6904bb28-9deb-4ac4-9e0d-64c2e0436e08 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:40.855905 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6904bb28-9deb-4ac4-9e0d-64c2e0436e08 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:40.881288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-6904bb28-9deb-4ac4-9e0d-64c2e0436e08 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:12:40.881288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:40.881288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:40.881288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:40.881288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:40.881288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:40.881288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:40.881288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:40.881288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:40.881288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:40.881288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:40.881288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:40.881288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:12:40.881288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:12:40.881288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:12:40.883368 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:12:40.883368 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:40.883368 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:40.883368 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:40.883368 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:40.883368 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:40.883368 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:40.883368 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:40.883368 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:40.883368 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:40.883368 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:40.883368 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:40.883368 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:40.883368 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:40.883368 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:40.883368 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:40.883368 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:40.885688 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:12:40.885688 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:12:40.885688 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:12:40.885688 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:12:40.885688 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:12:40.885688 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:12:40.885688 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:12:40.885688 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:12:40.885688 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 269/520] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:12:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 399 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:40.885688 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6904bb28-9deb-4ac4-9e0d-64c2e0436e08 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:40.992405 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9ad7e4-eb6f-45dd-a69f-70357deec90d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:41.056008 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daed137-c9a9-460c-a98f-cf70060bbacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:41.056702 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daed137-c9a9-460c-a98f-cf70060bbacc None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:41.056963 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daed137-c9a9-460c-a98f-cf70060bbacc None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:41.057525 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daed137-c9a9-460c-a98f-cf70060bbacc None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:41.058305 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daed137-c9a9-460c-a98f-cf70060bbacc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:41.058714 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daed137-c9a9-460c-a98f-cf70060bbacc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:41.060500 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daed137-c9a9-460c-a98f-cf70060bbacc None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:41.090285 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6904bb28-9deb-4ac4-9e0d-64c2e0436e08 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:41.092167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 252/521] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:12:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:41.092167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daed137-c9a9-460c-a98f-cf70060bbacc None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:41.162848 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcedb9f-55f1-4e8e-b398-73698187b8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:41.164074 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcedb9f-55f1-4e8e-b398-73698187b8ef None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:41.164520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcedb9f-55f1-4e8e-b398-73698187b8ef None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:41.164955 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcedb9f-55f1-4e8e-b398-73698187b8ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:41.178953 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcedb9f-55f1-4e8e-b398-73698187b8ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:41.181070 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcedb9f-55f1-4e8e-b398-73698187b8ef None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:41.207823 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-7dcedb9f-55f1-4e8e-b398-73698187b8ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:12:41.207823 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:41.207823 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:41.207823 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:41.207823 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:41.207823 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:41.207823 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:41.207823 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:41.207823 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:41.207823 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:41.207823 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:41.207823 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:41.207823 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:12:41.207823 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:12:41.207823 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:12:41.209470 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:12:41.209470 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:41.209470 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:41.209470 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:41.209470 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:41.209470 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:41.209470 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:41.209470 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:41.209470 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:41.209470 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:41.209470 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:41.209470 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:41.209470 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:41.209470 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:41.209470 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:41.209470 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:41.209470 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:41.210932 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:12:41.210932 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:12:41.210932 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:12:41.210932 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:12:41.210932 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:12:41.210932 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:12:41.210932 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:12:41.210932 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:12:41.210932 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 270/522] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:12:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:41.211637 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcedb9f-55f1-4e8e-b398-73698187b8ef None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:41.260878 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:41.292042 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:41.333116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daed137-c9a9-460c-a98f-cf70060bbacc None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:41.396331 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3424a2e-2a27-4184-bd33-8fc368fc998c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:41.397210 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3424a2e-2a27-4184-bd33-8fc368fc998c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:41.397516 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3424a2e-2a27-4184-bd33-8fc368fc998c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:41.398767 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3424a2e-2a27-4184-bd33-8fc368fc998c None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:41.399823 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3424a2e-2a27-4184-bd33-8fc368fc998c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:41.400560 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3424a2e-2a27-4184-bd33-8fc368fc998c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:41.402580 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3424a2e-2a27-4184-bd33-8fc368fc998c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:41.430369 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcedb9f-55f1-4e8e-b398-73698187b8ef None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:41.467241 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 253/523] 10.222.0.87 () {56 vars in 1137 bytes} [Tue Jun 21 18:12:41 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:41.467241 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3424a2e-2a27-4184-bd33-8fc368fc998c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:41.503370 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e0466a-fb8d-40d8-a5e5-dca0293bcfe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:41.504996 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e0466a-fb8d-40d8-a5e5-dca0293bcfe5 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:41.505598 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e0466a-fb8d-40d8-a5e5-dca0293bcfe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:41.506207 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e0466a-fb8d-40d8-a5e5-dca0293bcfe5 None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:41.508083 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e0466a-fb8d-40d8-a5e5-dca0293bcfe5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:41.508083 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e0466a-fb8d-40d8-a5e5-dca0293bcfe5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:41.510011 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e0466a-fb8d-40d8-a5e5-dca0293bcfe5 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:41.540496 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 271/524] 10.222.0.87 () {56 vars in 1148 bytes} [Tue Jun 21 18:12:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:41.541770 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e0466a-fb8d-40d8-a5e5-dca0293bcfe5 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:41.663656 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3424a2e-2a27-4184-bd33-8fc368fc998c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:41.698905 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:41.706543 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e0466a-fb8d-40d8-a5e5-dca0293bcfe5 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:41.751434 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:41.765898 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8bc293-4872-40d4-b58e-54bde10c2ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:41.767466 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8bc293-4872-40d4-b58e-54bde10c2ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:41.767981 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8bc293-4872-40d4-b58e-54bde10c2ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:41.768657 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8bc293-4872-40d4-b58e-54bde10c2ac7 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:41.775572 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d7c332-0673-4fa7-a58d-7ffc51eb29de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:41.776427 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d7c332-0673-4fa7-a58d-7ffc51eb29de None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:41.776835 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d7c332-0673-4fa7-a58d-7ffc51eb29de None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:41.777254 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d7c332-0673-4fa7-a58d-7ffc51eb29de None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:41.778332 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d7c332-0673-4fa7-a58d-7ffc51eb29de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:41.778777 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d7c332-0673-4fa7-a58d-7ffc51eb29de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:41.780444 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d7c332-0673-4fa7-a58d-7ffc51eb29de None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:41.784155 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8bc293-4872-40d4-b58e-54bde10c2ac7 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:41.787030 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8bc293-4872-40d4-b58e-54bde10c2ac7 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:41.807346 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 272/525] 10.222.0.87 () {56 vars in 1148 bytes} [Tue Jun 21 18:12:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:41.807346 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d7c332-0673-4fa7-a58d-7ffc51eb29de None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:41.816583 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-0e8bc293-4872-40d4-b58e-54bde10c2ac7 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 21 18:12:41.816583 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:41.816583 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:41.816583 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:41.816583 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:41.816583 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:41.816583 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:41.816583 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:41.816583 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:41.816583 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:41.816583 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:41.816583 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:41.816583 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:12:41.816583 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:12:41.816583 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:12:41.816583 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:12:41.818675 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:41.818675 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:41.818675 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:12:41.818675 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:12:41.818675 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:12:41.818675 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:12:41.818675 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:41.818675 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:41.818675 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:41.818675 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:41.818675 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:41.818675 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:41.818675 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:41.818675 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:41.818675 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:41.818675 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:41.818675 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:41.818675 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:41.823020 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:41.823020 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:41.823020 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:12:41.823020 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:12:41.823020 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:12:41.823020 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:12:41.823020 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:12:41.823020 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:12:41.823020 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 21 18:12:41.823020 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:12:41.823020 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 254/526] 10.222.0.87 () {56 vars in 1102 bytes} [Tue Jun 21 18:12:41 2022] GET /identity/v3/users/neutron => generated 85 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:41.823020 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8bc293-4872-40d4-b58e-54bde10c2ac7 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:42.023094 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d7c332-0673-4fa7-a58d-7ffc51eb29de None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:42.056010 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8bc293-4872-40d4-b58e-54bde10c2ac7 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:42.083262 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc0e384-ed9d-4268-9a93-7dda7413919f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:42.084381 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc0e384-ed9d-4268-9a93-7dda7413919f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:42.084913 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc0e384-ed9d-4268-9a93-7dda7413919f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:42.085433 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc0e384-ed9d-4268-9a93-7dda7413919f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:42.134119 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dae3c1-5a51-4ff7-85fc-2b66a50337ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:42.135138 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dae3c1-5a51-4ff7-85fc-2b66a50337ad None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:42.135586 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dae3c1-5a51-4ff7-85fc-2b66a50337ad None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:42.136027 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dae3c1-5a51-4ff7-85fc-2b66a50337ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:42.137510 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:42.138156 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dae3c1-5a51-4ff7-85fc-2b66a50337ad None admin] RBAC: Adding query filter params (role.id=b9dbf55e6f2947158c1d946f6ee9f907, scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb, user.id=a1b3fa67905c4027b3656ac36e2b76c9) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:42.138156 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dae3c1-5a51-4ff7-85fc-2b66a50337ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:42.141579 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dae3c1-5a51-4ff7-85fc-2b66a50337ad None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:42.161676 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 255/527] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:12:41 2022] GET /identity/v3/role_assignments?user.id=a1b3fa67905c4027b3656ac36e2b76c9&scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb&role.id=b9dbf55e6f2947158c1d946f6ee9f907 => generated 259 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:42.161676 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dae3c1-5a51-4ff7-85fc-2b66a50337ad None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:42.246243 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc0e384-ed9d-4268-9a93-7dda7413919f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:42.250385 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc0e384-ed9d-4268-9a93-7dda7413919f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:42.256945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:42.343917 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dae3c1-5a51-4ff7-85fc-2b66a50337ad None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:42.424472 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ffea8b-7b79-40b1-b7a2-793ad22bfe97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:42.425719 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ffea8b-7b79-40b1-b7a2-793ad22bfe97 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:42.426166 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ffea8b-7b79-40b1-b7a2-793ad22bfe97 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:42.426686 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ffea8b-7b79-40b1-b7a2-793ad22bfe97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:42.428053 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ffea8b-7b79-40b1-b7a2-793ad22bfe97 None admin] RBAC: Adding query filter params (role.id=b9dbf55e6f2947158c1d946f6ee9f907, scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb, user.id=e7da4bf6f3be47b4aa25c6240cb4be03) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:42.428620 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ffea8b-7b79-40b1-b7a2-793ad22bfe97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:42.430941 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ffea8b-7b79-40b1-b7a2-793ad22bfe97 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:42.443497 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 256/528] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:12:42 2022] GET /identity/v3/role_assignments?user.id=e7da4bf6f3be47b4aa25c6240cb4be03&scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb&role.id=b9dbf55e6f2947158c1d946f6ee9f907 => generated 259 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:42.446860 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ffea8b-7b79-40b1-b7a2-793ad22bfe97 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:42.488838 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 273/529] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:12:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 690 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:12:42.499701 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc0e384-ed9d-4268-9a93-7dda7413919f None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:42.592195 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ffea8b-7b79-40b1-b7a2-793ad22bfe97 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:42.665780 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:42.708692 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30a7575-8e9e-470e-aa5a-a13fa151e3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:42.709915 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a7575-8e9e-470e-aa5a-a13fa151e3d2 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:42.710072 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:42.710422 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a7575-8e9e-470e-aa5a-a13fa151e3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:42.710888 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a7575-8e9e-470e-aa5a-a13fa151e3d2 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:42.712363 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30a7575-8e9e-470e-aa5a-a13fa151e3d2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:42.712871 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30a7575-8e9e-470e-aa5a-a13fa151e3d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:42.715530 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30a7575-8e9e-470e-aa5a-a13fa151e3d2 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:42.783979 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 257/530] 10.222.0.87 () {56 vars in 1139 bytes} [Tue Jun 21 18:12:42 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:42.796498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30a7575-8e9e-470e-aa5a-a13fa151e3d2 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:42.880956 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc0e384-ed9d-4268-9a93-7dda7413919f None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:42.953414 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30a7575-8e9e-470e-aa5a-a13fa151e3d2 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:43.002945 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3c7a5e-4080-4d50-a747-a9bcc28aa30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:43.004394 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c7a5e-4080-4d50-a747-a9bcc28aa30b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:43.004980 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c7a5e-4080-4d50-a747-a9bcc28aa30b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:43.005460 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c7a5e-4080-4d50-a747-a9bcc28aa30b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:43.010152 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1531362e-685e-49c9-a145-9b13e3b660b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:43.011879 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531362e-685e-49c9-a145-9b13e3b660b2 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:43.012815 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531362e-685e-49c9-a145-9b13e3b660b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:43.013680 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531362e-685e-49c9-a145-9b13e3b660b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:43.044261 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3c7a5e-4080-4d50-a747-a9bcc28aa30b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:43.048557 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3c7a5e-4080-4d50-a747-a9bcc28aa30b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:43.090966 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-9b3c7a5e-4080-4d50-a747-a9bcc28aa30b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:43.090966 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:43.090966 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:43.090966 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:43.090966 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:43.090966 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:43.090966 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:43.090966 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:43.090966 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:43.090966 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:43.090966 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:43.090966 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:43.090966 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:12:43.090966 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:12:43.090966 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:12:43.093371 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:12:43.093371 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:43.093371 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:43.093371 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:43.093371 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:43.093371 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:43.093371 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:43.093371 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:43.093371 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:43.093371 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:43.093371 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:43.093371 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:43.093371 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:43.093371 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:43.093371 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:43.093371 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:43.093371 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:43.095160 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:12:43.095160 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:12:43.095160 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:43.095160 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:12:43.099840 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 274/531] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:12:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 601 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:12:43.111366 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b0807-5bf7-4894-8c8c-0fb35e0aa976 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:43.112695 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:43.113596 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b0807-5bf7-4894-8c8c-0fb35e0aa976 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:43.115707 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b0807-5bf7-4894-8c8c-0fb35e0aa976 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:43.121922 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 275/532] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:12:43 2022] GET /identity => generated 270 bytes in 21 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:12:43.123982 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2b0807-5bf7-4894-8c8c-0fb35e0aa976 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:43.146021 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1531362e-685e-49c9-a145-9b13e3b660b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:43.148208 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1531362e-685e-49c9-a145-9b13e3b660b2 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:43.303675 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 258/533] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:12:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:12:43.303675 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ecb58-a4b3-4a29-9a0a-6ef1f7dd7521 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:43.303675 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ecb58-a4b3-4a29-9a0a-6ef1f7dd7521 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:43.303675 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ecb58-a4b3-4a29-9a0a-6ef1f7dd7521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:43.324636 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d58ecb58-a4b3-4a29-9a0a-6ef1f7dd7521 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:12:43.326358 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d58ecb58-a4b3-4a29-9a0a-6ef1f7dd7521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:12:43.356591 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2b0807-5bf7-4894-8c8c-0fb35e0aa976 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:43.384029 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:43.432649 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f718a3d4-2758-4669-bbc9-213efcf74291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:43.434288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718a3d4-2758-4669-bbc9-213efcf74291 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:43.435825 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718a3d4-2758-4669-bbc9-213efcf74291 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:43.436200 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 259/534] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:12:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:12:43.437835 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ecb58-a4b3-4a29-9a0a-6ef1f7dd7521 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:43.442571 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718a3d4-2758-4669-bbc9-213efcf74291 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:43.444381 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f718a3d4-2758-4669-bbc9-213efcf74291 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:43.444749 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f718a3d4-2758-4669-bbc9-213efcf74291 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:43.447603 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f718a3d4-2758-4669-bbc9-213efcf74291 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:43.458266 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:43.501588 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 276/535] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:12:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:43.510564 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f718a3d4-2758-4669-bbc9-213efcf74291 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:43.599386 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ecb58-a4b3-4a29-9a0a-6ef1f7dd7521 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:43.654133 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f80438-6a0e-47ea-8707-9f22d62c0017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:43.655455 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f80438-6a0e-47ea-8707-9f22d62c0017 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:43.656116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f80438-6a0e-47ea-8707-9f22d62c0017 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:43.656710 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f80438-6a0e-47ea-8707-9f22d62c0017 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:43.673174 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f80438-6a0e-47ea-8707-9f22d62c0017 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:43.675574 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f80438-6a0e-47ea-8707-9f22d62c0017 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:43.701122 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-00f80438-6a0e-47ea-8707-9f22d62c0017 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:43.701122 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:43.701122 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:43.701122 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:43.701122 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:43.701122 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:43.701122 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:43.701122 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:43.701122 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:43.701122 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:43.701122 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:43.701122 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:43.701122 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:12:43.701122 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:12:43.701122 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:12:43.702131 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:12:43.702131 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:43.702131 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:43.702131 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:43.702131 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:43.702131 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:43.702131 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:43.702131 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:43.702131 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:43.702131 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:43.702131 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:43.702131 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:43.702131 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:43.702131 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:43.702131 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:43.702131 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:43.702131 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:43.703749 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:12:43.703749 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:12:43.703749 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:43.703749 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:12:43.707364 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 260/536] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:12:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:43.708706 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f80438-6a0e-47ea-8707-9f22d62c0017 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:43.755545 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f718a3d4-2758-4669-bbc9-213efcf74291 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:43.822038 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:43.833087 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc80ba-ff98-4658-8c51-a6c399e9cae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:43.835197 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc80ba-ff98-4658-8c51-a6c399e9cae6 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:43.836106 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc80ba-ff98-4658-8c51-a6c399e9cae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:43.836905 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc80ba-ff98-4658-8c51-a6c399e9cae6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:43.856094 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bc80ba-ff98-4658-8c51-a6c399e9cae6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:43.859634 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bc80ba-ff98-4658-8c51-a6c399e9cae6 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:43.889532 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-82bc80ba-ff98-4658-8c51-a6c399e9cae6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:12:43.889532 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:43.889532 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:43.889532 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:43.889532 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:43.889532 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:43.889532 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:43.889532 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:43.889532 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:43.889532 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:43.889532 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:43.889532 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:43.889532 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:12:43.889532 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:12:43.889532 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:12:43.891178 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:12:43.891178 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:43.891178 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:43.891178 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:43.891178 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:43.891178 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:43.891178 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:43.891178 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:43.891178 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:43.891178 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:43.891178 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:43.891178 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:43.891178 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:43.891178 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:43.891178 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:43.891178 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:43.891178 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:43.892725 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:12:43.892725 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:12:43.892725 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:12:43.892725 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:12:43.892725 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:12:43.892725 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:12:43.892725 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:12:43.892725 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:12:43.897106 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 277/537] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:12:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 388 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:43.899046 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc80ba-ff98-4658-8c51-a6c399e9cae6 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:43.918756 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f80438-6a0e-47ea-8707-9f22d62c0017 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:43.959118 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:43.986556 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923a7a5-8c2e-4894-b57c-43312fde6039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:43.987662 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923a7a5-8c2e-4894-b57c-43312fde6039 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:43.988240 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923a7a5-8c2e-4894-b57c-43312fde6039 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:43.988736 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923a7a5-8c2e-4894-b57c-43312fde6039 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:44.010809 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b923a7a5-8c2e-4894-b57c-43312fde6039 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:12:44.027572 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-b923a7a5-8c2e-4894-b57c-43312fde6039 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:12:44.027572 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:44.027572 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:44.027572 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:44.027572 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:44.027572 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:44.027572 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:44.027572 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:44.027572 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:44.027572 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:44.027572 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:44.027572 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:44.027572 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:12:44.027572 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:12:44.027572 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:12:44.027572 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:12:44.029381 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:12:44.029381 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:12:44.029381 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:12:44.029381 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:12:44.029381 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:44.029381 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:44.029381 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:44.029381 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:44.029381 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:44.029381 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:44.029381 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:44.029381 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:44.029381 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:44.029381 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:44.029381 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:44.029381 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:44.030950 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:44.030950 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:44.030950 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:44.030950 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:44.030950 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:12:44.030950 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:12:44.030950 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:12:44.030950 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:12:44.030950 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:12:44.030950 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:12:44.030950 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:12:44.030950 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:12:44.030950 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 261/538] 10.222.0.87 () {56 vars in 1102 bytes} [Tue Jun 21 18:12:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:44.033201 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923a7a5-8c2e-4894-b57c-43312fde6039 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:44.107639 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc80ba-ff98-4658-8c51-a6c399e9cae6 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:44.162670 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece8dfae-6a91-49b0-aa67-1b3d22fd712b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:44.163886 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece8dfae-6a91-49b0-aa67-1b3d22fd712b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:44.164506 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece8dfae-6a91-49b0-aa67-1b3d22fd712b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:44.164871 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece8dfae-6a91-49b0-aa67-1b3d22fd712b None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:44.166477 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece8dfae-6a91-49b0-aa67-1b3d22fd712b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:44.167041 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece8dfae-6a91-49b0-aa67-1b3d22fd712b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:44.172051 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece8dfae-6a91-49b0-aa67-1b3d22fd712b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:44.202058 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 278/539] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:12:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:44.203665 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece8dfae-6a91-49b0-aa67-1b3d22fd712b None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:44.233905 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923a7a5-8c2e-4894-b57c-43312fde6039 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:44.277514 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:44.308281 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56390ee2-1ba5-4153-a0f0-dcbd0ad43525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:44.309493 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56390ee2-1ba5-4153-a0f0-dcbd0ad43525 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:44.310132 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56390ee2-1ba5-4153-a0f0-dcbd0ad43525 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:44.310648 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56390ee2-1ba5-4153-a0f0-dcbd0ad43525 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:44.313106 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56390ee2-1ba5-4153-a0f0-dcbd0ad43525 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:44.313106 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56390ee2-1ba5-4153-a0f0-dcbd0ad43525 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:44.315780 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56390ee2-1ba5-4153-a0f0-dcbd0ad43525 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:44.366594 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 262/540] 10.222.0.87 () {56 vars in 1148 bytes} [Tue Jun 21 18:12:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:44.368108 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56390ee2-1ba5-4153-a0f0-dcbd0ad43525 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:44.385300 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:44.404478 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece8dfae-6a91-49b0-aa67-1b3d22fd712b None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:44.471348 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0b3436-d575-4b8c-bcf7-e4a7da8c3c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:44.473674 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0b3436-d575-4b8c-bcf7-e4a7da8c3c79 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:44.474232 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0b3436-d575-4b8c-bcf7-e4a7da8c3c79 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:44.474716 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0b3436-d575-4b8c-bcf7-e4a7da8c3c79 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:44.476363 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0b3436-d575-4b8c-bcf7-e4a7da8c3c79 None admin] RBAC: Adding query filter params (name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:44.476739 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0b3436-d575-4b8c-bcf7-e4a7da8c3c79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:44.479905 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0b3436-d575-4b8c-bcf7-e4a7da8c3c79 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:44.497597 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 279/541] 10.222.0.87 () {56 vars in 1103 bytes} [Tue Jun 21 18:12:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:44.499217 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0b3436-d575-4b8c-bcf7-e4a7da8c3c79 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:44.561101 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56390ee2-1ba5-4153-a0f0-dcbd0ad43525 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:44.608126 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f394d7d-a290-4148-96a5-18f831987a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:44.610006 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f394d7d-a290-4148-96a5-18f831987a9f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:44.610861 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f394d7d-a290-4148-96a5-18f831987a9f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:44.611608 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f394d7d-a290-4148-96a5-18f831987a9f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:44.627829 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f394d7d-a290-4148-96a5-18f831987a9f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:44.631658 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f394d7d-a290-4148-96a5-18f831987a9f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:44.667262 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-9f394d7d-a290-4148-96a5-18f831987a9f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:12:44.667262 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:44.667262 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:44.667262 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:44.667262 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:44.667262 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:44.667262 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:44.667262 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:44.667262 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:44.667262 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:44.667262 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:44.667262 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:44.667262 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:12:44.667262 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:12:44.667262 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:12:44.668945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:12:44.668945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:44.668945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:44.668945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:44.668945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:44.668945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:44.668945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:44.668945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:44.668945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:44.668945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:44.668945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:44.668945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:44.668945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:44.668945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:44.668945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:44.668945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:44.668945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:44.670626 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:12:44.670626 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:12:44.670626 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:12:44.670626 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:12:44.670626 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:12:44.670626 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:12:44.670626 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:12:44.670626 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:12:44.677712 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 263/542] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:12:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:44.677712 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f394d7d-a290-4148-96a5-18f831987a9f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:44.756218 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0b3436-d575-4b8c-bcf7-e4a7da8c3c79 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:44.765224 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:44.850107 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884b4e45-8190-41a6-89ab-96608ef1f57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:44.851840 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b4e45-8190-41a6-89ab-96608ef1f57b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:44.852609 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b4e45-8190-41a6-89ab-96608ef1f57b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:44.853369 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b4e45-8190-41a6-89ab-96608ef1f57b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:44.854271 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884b4e45-8190-41a6-89ab-96608ef1f57b None admin] RBAC: Adding query filter params (role.id=b9dbf55e6f2947158c1d946f6ee9f907, scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb, user.id=b683e6f78abd4986b415eb9ddf34cb8d) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:44.854715 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884b4e45-8190-41a6-89ab-96608ef1f57b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:44.857721 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884b4e45-8190-41a6-89ab-96608ef1f57b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:44.868108 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 280/543] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:12:44 2022] GET /identity/v3/role_assignments?user.id=b683e6f78abd4986b415eb9ddf34cb8d&scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb&role.id=b9dbf55e6f2947158c1d946f6ee9f907 => generated 259 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:44.869203 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884b4e45-8190-41a6-89ab-96608ef1f57b None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:44.906245 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:44.909535 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f394d7d-a290-4148-96a5-18f831987a9f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:44.972772 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f46c4f-eddf-45a8-b67f-6a87e785480e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:44.975542 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f46c4f-eddf-45a8-b67f-6a87e785480e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:44.976211 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f46c4f-eddf-45a8-b67f-6a87e785480e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:44.976847 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f46c4f-eddf-45a8-b67f-6a87e785480e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:44.999417 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f46c4f-eddf-45a8-b67f-6a87e785480e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:45.002781 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f46c4f-eddf-45a8-b67f-6a87e785480e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:45.059138 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-78f46c4f-eddf-45a8-b67f-6a87e785480e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:45.059138 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:45.059138 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:45.059138 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:45.059138 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:45.059138 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:45.059138 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:45.059138 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:45.059138 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:45.059138 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:45.059138 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:45.059138 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:45.059138 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:12:45.059138 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:12:45.059138 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:12:45.060775 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:12:45.060775 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:45.060775 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:45.060775 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:45.060775 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:45.060775 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:45.060775 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:45.060775 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:45.060775 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:45.060775 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:45.060775 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:45.060775 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:45.060775 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:45.060775 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:45.060775 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:45.060775 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:45.060775 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:45.062589 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:12:45.062589 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:12:45.062589 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:45.062589 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:12:45.074041 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 264/544] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:12:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 401 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:45.081338 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f46c4f-eddf-45a8-b67f-6a87e785480e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:45.086654 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884b4e45-8190-41a6-89ab-96608ef1f57b None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:45.165462 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5402b794-4895-4c3d-8e12-8ab68c09d842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:45.169680 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402b794-4895-4c3d-8e12-8ab68c09d842 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:45.169680 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402b794-4895-4c3d-8e12-8ab68c09d842 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:45.169680 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402b794-4895-4c3d-8e12-8ab68c09d842 None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:45.169680 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5402b794-4895-4c3d-8e12-8ab68c09d842 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:45.169680 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5402b794-4895-4c3d-8e12-8ab68c09d842 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:45.171921 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5402b794-4895-4c3d-8e12-8ab68c09d842 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:45.196760 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 281/545] 10.222.0.87 () {56 vars in 1148 bytes} [Tue Jun 21 18:12:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:45.203103 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5402b794-4895-4c3d-8e12-8ab68c09d842 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:45.330217 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:45.368131 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:45.399763 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f46c4f-eddf-45a8-b67f-6a87e785480e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:45.408184 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5402b794-4895-4c3d-8e12-8ab68c09d842 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:45.498452 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc9fd7-08e4-4f7c-ba16-0b446fd5842a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:45.499589 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc9fd7-08e4-4f7c-ba16-0b446fd5842a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:45.500084 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc9fd7-08e4-4f7c-ba16-0b446fd5842a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:45.500577 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc9fd7-08e4-4f7c-ba16-0b446fd5842a None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:45.502028 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bc9fd7-08e4-4f7c-ba16-0b446fd5842a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:45.502573 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bc9fd7-08e4-4f7c-ba16-0b446fd5842a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:45.505252 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bc9fd7-08e4-4f7c-ba16-0b446fd5842a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:45.529925 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 265/546] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:12:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 449 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:12:45.538751 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc9fd7-08e4-4f7c-ba16-0b446fd5842a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:45.540414 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84f053a-4818-4a07-a922-dedb40a74749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:45.548532 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f053a-4818-4a07-a922-dedb40a74749 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:45.549251 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f053a-4818-4a07-a922-dedb40a74749 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:45.549794 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f053a-4818-4a07-a922-dedb40a74749 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:45.554575 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84f053a-4818-4a07-a922-dedb40a74749 None admin] RBAC: Adding query filter params (role.id=b9dbf55e6f2947158c1d946f6ee9f907, scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb, user.id=480e2529015c4c2fb4d6cd1985009fde) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:45.555121 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84f053a-4818-4a07-a922-dedb40a74749 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:45.563402 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84f053a-4818-4a07-a922-dedb40a74749 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:45.587100 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 282/547] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:12:45 2022] GET /identity/v3/role_assignments?user.id=480e2529015c4c2fb4d6cd1985009fde&scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb&role.id=b9dbf55e6f2947158c1d946f6ee9f907 => generated 259 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:12:45.720411 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5355ab-16ce-43b7-b234-6723ddd2f88e None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:45.721127 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5355ab-16ce-43b7-b234-6723ddd2f88e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:45.721857 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5355ab-16ce-43b7-b234-6723ddd2f88e None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:45.729851 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 283/548] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:12:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:12:45.737706 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b073b3-63e2-4d2b-8824-b29177dfb718 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:45.737706 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b073b3-63e2-4d2b-8824-b29177dfb718 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:45.737706 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b073b3-63e2-4d2b-8824-b29177dfb718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:45.794445 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-d7b073b3-63e2-4d2b-8824-b29177dfb718 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:12:45.800147 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d7b073b3-63e2-4d2b-8824-b29177dfb718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:12:45.836655 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc9fd7-08e4-4f7c-ba16-0b446fd5842a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:45.940674 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec20c7-e0cd-4c0a-bf12-2ca17c7b63e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:45.942596 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec20c7-e0cd-4c0a-bf12-2ca17c7b63e7 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:45.942777 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec20c7-e0cd-4c0a-bf12-2ca17c7b63e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:45.943199 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec20c7-e0cd-4c0a-bf12-2ca17c7b63e7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:45.977936 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ec20c7-e0cd-4c0a-bf12-2ca17c7b63e7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:45.981018 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ec20c7-e0cd-4c0a-bf12-2ca17c7b63e7 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:45.998030 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 284/549] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:12:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:12:46.018952 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-80ec20c7-e0cd-4c0a-bf12-2ca17c7b63e7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 21 18:12:46.018952 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:46.018952 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:46.018952 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:46.018952 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:46.018952 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:46.018952 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:46.018952 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:46.018952 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:46.018952 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:46.018952 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:46.018952 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:46.018952 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:12:46.018952 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:12:46.018952 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:12:46.018952 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:12:46.021167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:46.021167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:46.021167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:12:46.021167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:12:46.021167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:12:46.021167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:12:46.021167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:46.021167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:46.021167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:46.021167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:46.021167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:46.021167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:46.021167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:46.021167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:46.021167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:46.021167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:46.021167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:46.021167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:46.023391 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:46.023391 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:46.023391 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:12:46.023391 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:12:46.023391 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:12:46.023391 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:12:46.023391 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:12:46.023391 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:12:46.023391 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 21 18:12:46.023391 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:12:46.023391 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 266/550] 10.222.0.87 () {56 vars in 1096 bytes} [Tue Jun 21 18:12:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 484 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:12:46.031897 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b073b3-63e2-4d2b-8824-b29177dfb718 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:46.053307 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec20c7-e0cd-4c0a-bf12-2ca17c7b63e7 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:46.094141 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:46.185389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:46.249707 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b073b3-63e2-4d2b-8824-b29177dfb718 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:46.272927 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec20c7-e0cd-4c0a-bf12-2ca17c7b63e7 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:46.317579 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4916c01a-a58c-4cd0-b665-ecff44c73963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:46.318791 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4916c01a-a58c-4cd0-b665-ecff44c73963 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:46.319198 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4916c01a-a58c-4cd0-b665-ecff44c73963 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:46.319697 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4916c01a-a58c-4cd0-b665-ecff44c73963 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:46.321319 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4916c01a-a58c-4cd0-b665-ecff44c73963 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:46.321922 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4916c01a-a58c-4cd0-b665-ecff44c73963 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:46.324744 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4916c01a-a58c-4cd0-b665-ecff44c73963 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:46.342262 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6ea428-c2a4-44ef-bbcc-9530a1d816f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:46.344293 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6ea428-c2a4-44ef-bbcc-9530a1d816f9 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:46.345045 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6ea428-c2a4-44ef-bbcc-9530a1d816f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:46.345781 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6ea428-c2a4-44ef-bbcc-9530a1d816f9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:46.362626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 285/551] 10.222.0.87 () {56 vars in 1135 bytes} [Tue Jun 21 18:12:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:46.365000 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4db477-36e2-4cd5-9892-a8dff93164e2 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:46.365451 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4db477-36e2-4cd5-9892-a8dff93164e2 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:46.365916 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4db477-36e2-4cd5-9892-a8dff93164e2 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:46.368917 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 286/552] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:12:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:12:46.372504 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4db477-36e2-4cd5-9892-a8dff93164e2 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:46.376874 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6ea428-c2a4-44ef-bbcc-9530a1d816f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:12:46.397201 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-ba6ea428-c2a4-44ef-bbcc-9530a1d816f9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:12:46.397201 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:46.397201 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:46.397201 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:46.397201 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:46.397201 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:46.397201 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:46.397201 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:46.397201 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:46.397201 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:46.397201 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:46.397201 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:46.397201 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:12:46.397201 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:12:46.397201 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:12:46.397201 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:12:46.399295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:12:46.399295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:12:46.399295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:12:46.399295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:12:46.399295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:46.399295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:46.399295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:46.399295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:46.399295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:46.399295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:46.399295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:46.399295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:46.399295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:46.399295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:46.399295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:46.399295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:46.400800 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:46.400800 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:46.400800 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:46.400800 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:46.400800 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:12:46.400800 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:12:46.400800 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:12:46.400800 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:12:46.400800 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:12:46.400800 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:12:46.400800 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:12:46.400800 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:12:46.403811 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 267/553] 10.222.0.87 () {56 vars in 1102 bytes} [Tue Jun 21 18:12:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:12:46.409104 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598d670-c9ef-49ee-97e1-45b5c70cd875 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:46.410205 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598d670-c9ef-49ee-97e1-45b5c70cd875 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:46.410979 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598d670-c9ef-49ee-97e1-45b5c70cd875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:46.442264 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a598d670-c9ef-49ee-97e1-45b5c70cd875 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:12:46.445113 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a598d670-c9ef-49ee-97e1-45b5c70cd875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:12:46.613759 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 268/554] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:12:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:12:46.615825 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a598d670-c9ef-49ee-97e1-45b5c70cd875 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:46.620971 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:46.624872 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4db477-36e2-4cd5-9892-a8dff93164e2 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:46.720842 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:46.741217 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2000797d-f2bb-4d3b-a9b6-57d944f2db6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:46.742793 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2000797d-f2bb-4d3b-a9b6-57d944f2db6a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:46.742968 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2000797d-f2bb-4d3b-a9b6-57d944f2db6a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:46.743435 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2000797d-f2bb-4d3b-a9b6-57d944f2db6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:46.898990 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2000797d-f2bb-4d3b-a9b6-57d944f2db6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:46.901742 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2000797d-f2bb-4d3b-a9b6-57d944f2db6a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:46.924988 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a598d670-c9ef-49ee-97e1-45b5c70cd875 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:47.006432 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fedce4-a213-482d-8027-1e847f0b8f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:47.007748 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fedce4-a213-482d-8027-1e847f0b8f93 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:47.008123 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fedce4-a213-482d-8027-1e847f0b8f93 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:47.008626 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fedce4-a213-482d-8027-1e847f0b8f93 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:47.010187 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fedce4-a213-482d-8027-1e847f0b8f93 None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:47.010756 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fedce4-a213-482d-8027-1e847f0b8f93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:47.013690 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fedce4-a213-482d-8027-1e847f0b8f93 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:47.032074 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 269/555] 10.222.0.87 () {56 vars in 1103 bytes} [Tue Jun 21 18:12:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:47.033724 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fedce4-a213-482d-8027-1e847f0b8f93 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:47.146250 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 287/556] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:12:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 775 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:12:47.148398 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2000797d-f2bb-4d3b-a9b6-57d944f2db6a None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:47.164869 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:47.211346 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:47.247359 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fedce4-a213-482d-8027-1e847f0b8f93 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:47.315163 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea4aa1-3246-4391-9ecc-2feed748f12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:47.316402 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea4aa1-3246-4391-9ecc-2feed748f12f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:47.316892 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea4aa1-3246-4391-9ecc-2feed748f12f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:47.317358 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea4aa1-3246-4391-9ecc-2feed748f12f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:47.339200 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ea4aa1-3246-4391-9ecc-2feed748f12f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:12:47.355418 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-96ea4aa1-3246-4391-9ecc-2feed748f12f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:12:47.355418 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:47.355418 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:47.355418 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:47.355418 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:47.355418 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:47.355418 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:47.355418 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:47.355418 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:47.355418 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:47.355418 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:47.355418 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:47.355418 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:12:47.355418 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:12:47.355418 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:12:47.355418 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:12:47.356892 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:12:47.356892 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:12:47.356892 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:12:47.356892 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:12:47.356892 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:47.356892 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:47.356892 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:47.356892 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:47.356892 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:47.356892 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:47.356892 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:47.356892 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:47.356892 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:47.356892 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:47.356892 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:47.356892 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:47.358173 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:47.358173 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:47.358173 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:47.358173 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:47.358173 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:12:47.358173 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:12:47.358173 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:12:47.358173 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:12:47.358173 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:12:47.358173 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:12:47.358173 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:12:47.358173 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:12:47.365819 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 270/557] 10.222.0.87 () {56 vars in 1102 bytes} [Tue Jun 21 18:12:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:47.367505 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea4aa1-3246-4391-9ecc-2feed748f12f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:47.369503 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2000797d-f2bb-4d3b-a9b6-57d944f2db6a None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:47.428083 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1714322e-3822-432c-b052-e70e9d6cd8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:47.430235 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1714322e-3822-432c-b052-e70e9d6cd8b2 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:47.431197 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1714322e-3822-432c-b052-e70e9d6cd8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:47.432041 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1714322e-3822-432c-b052-e70e9d6cd8b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:47.452062 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1714322e-3822-432c-b052-e70e9d6cd8b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:47.455368 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1714322e-3822-432c-b052-e70e9d6cd8b2 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:47.490476 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-1714322e-3822-432c-b052-e70e9d6cd8b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:47.490476 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:47.490476 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:47.490476 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:47.490476 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:47.490476 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:47.490476 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:47.490476 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:47.490476 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:47.490476 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:47.490476 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:47.490476 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:47.490476 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:12:47.490476 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:12:47.490476 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:12:47.492670 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:12:47.492670 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:47.492670 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:47.492670 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:47.492670 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:47.492670 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:47.492670 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:47.492670 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:47.492670 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:47.492670 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:47.492670 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:47.492670 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:47.492670 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:47.492670 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:47.492670 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:47.492670 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:47.492670 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:47.493564 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:12:47.493564 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:12:47.493564 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:47.493564 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:12:47.497174 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 288/558] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:12:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:47.498994 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1714322e-3822-432c-b052-e70e9d6cd8b2 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:47.583818 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea4aa1-3246-4391-9ecc-2feed748f12f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:47.633745 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:47.650412 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:47.657341 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81639784-d1fc-4d23-8486-741008132c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:47.658976 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81639784-d1fc-4d23-8486-741008132c18 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:47.659325 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81639784-d1fc-4d23-8486-741008132c18 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:47.659845 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81639784-d1fc-4d23-8486-741008132c18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:47.683165 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81639784-d1fc-4d23-8486-741008132c18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:47.686122 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81639784-d1fc-4d23-8486-741008132c18 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:47.691588 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1714322e-3822-432c-b052-e70e9d6cd8b2 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:47.722106 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-81639784-d1fc-4d23-8486-741008132c18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:47.722106 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:47.722106 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:47.722106 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:47.722106 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:47.722106 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:47.722106 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:47.722106 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:47.722106 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:47.722106 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:47.722106 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:47.722106 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:47.722106 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:12:47.722106 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:12:47.722106 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:12:47.723926 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:12:47.723926 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:47.723926 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:47.723926 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:47.723926 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:47.723926 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:47.723926 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:47.723926 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:47.723926 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:47.723926 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:47.723926 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:47.723926 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:47.723926 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:47.723926 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:47.723926 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:47.723926 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:47.723926 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:47.725260 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:12:47.725260 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:12:47.725260 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:47.725260 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:12:47.725260 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 271/559] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:12:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:47.729767 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81639784-d1fc-4d23-8486-741008132c18 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:47.769523 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6486e-3b4b-41cf-a5d5-d91c13afc4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:47.770888 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6486e-3b4b-41cf-a5d5-d91c13afc4b6 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:47.771400 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6486e-3b4b-41cf-a5d5-d91c13afc4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:47.771900 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6486e-3b4b-41cf-a5d5-d91c13afc4b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:47.773226 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de6486e-3b4b-41cf-a5d5-d91c13afc4b6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:47.773864 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de6486e-3b4b-41cf-a5d5-d91c13afc4b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:47.776390 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de6486e-3b4b-41cf-a5d5-d91c13afc4b6 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:47.801020 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 289/560] 10.222.0.87 () {56 vars in 1103 bytes} [Tue Jun 21 18:12:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:47.801020 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6486e-3b4b-41cf-a5d5-d91c13afc4b6 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:47.981377 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6486e-3b4b-41cf-a5d5-d91c13afc4b6 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:47.988899 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81639784-d1fc-4d23-8486-741008132c18 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:48.036813 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9180ea40-a0e0-45f1-bddf-4425135f49ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:48.038298 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180ea40-a0e0-45f1-bddf-4425135f49ea None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:48.038716 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180ea40-a0e0-45f1-bddf-4425135f49ea None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:48.039213 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180ea40-a0e0-45f1-bddf-4425135f49ea None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:48.040701 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9180ea40-a0e0-45f1-bddf-4425135f49ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:48.041391 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9180ea40-a0e0-45f1-bddf-4425135f49ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:48.043106 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9180ea40-a0e0-45f1-bddf-4425135f49ea None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:48.068387 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375ca359-10ad-4a96-ac90-2d93d3f33128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:48.071045 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ca359-10ad-4a96-ac90-2d93d3f33128 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:48.072030 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ca359-10ad-4a96-ac90-2d93d3f33128 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:48.072943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 290/561] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:12:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:48.073683 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ca359-10ad-4a96-ac90-2d93d3f33128 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:48.074778 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9180ea40-a0e0-45f1-bddf-4425135f49ea None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:48.076347 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375ca359-10ad-4a96-ac90-2d93d3f33128 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:48.077339 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375ca359-10ad-4a96-ac90-2d93d3f33128 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:48.079604 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:48.080767 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375ca359-10ad-4a96-ac90-2d93d3f33128 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:48.110060 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 272/562] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:12:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:48.112057 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375ca359-10ad-4a96-ac90-2d93d3f33128 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:48.189621 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:48.292577 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9180ea40-a0e0-45f1-bddf-4425135f49ea None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:48.359239 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d28e75-8da1-40a9-8393-01740f1cb772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:48.360042 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375ca359-10ad-4a96-ac90-2d93d3f33128 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:48.361060 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d28e75-8da1-40a9-8393-01740f1cb772 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:48.361521 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d28e75-8da1-40a9-8393-01740f1cb772 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:48.362039 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d28e75-8da1-40a9-8393-01740f1cb772 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:48.385684 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d28e75-8da1-40a9-8393-01740f1cb772 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:48.388856 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d28e75-8da1-40a9-8393-01740f1cb772 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:48.426283 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-f5d28e75-8da1-40a9-8393-01740f1cb772 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:48.426283 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:48.426283 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:48.426283 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:48.426283 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:48.426283 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:48.426283 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:48.426283 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:48.426283 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:48.426283 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:48.426283 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:48.426283 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:48.426283 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:12:48.426283 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:12:48.426283 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:12:48.428188 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:12:48.428188 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:48.428188 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:48.428188 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:48.428188 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:48.428188 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:48.428188 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:48.428188 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:48.428188 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:48.428188 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:48.428188 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:48.428188 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:48.428188 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:48.428188 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:48.428188 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:48.428188 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:48.428188 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:48.430128 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:12:48.430128 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:12:48.430128 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:48.430128 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:12:48.434423 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 291/563] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:12:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:48.436136 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d28e75-8da1-40a9-8393-01740f1cb772 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:48.457999 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0f66c8-3d0a-4512-a7a3-671e2730999b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:48.459935 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f66c8-3d0a-4512-a7a3-671e2730999b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:48.460748 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f66c8-3d0a-4512-a7a3-671e2730999b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:48.461589 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f66c8-3d0a-4512-a7a3-671e2730999b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:48.476703 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0f66c8-3d0a-4512-a7a3-671e2730999b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:48.480402 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0f66c8-3d0a-4512-a7a3-671e2730999b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:48.509305 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-bb0f66c8-3d0a-4512-a7a3-671e2730999b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:12:48.509305 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:48.509305 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:48.509305 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:48.509305 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:48.509305 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:48.509305 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:48.509305 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:48.509305 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:48.509305 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:48.509305 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:48.509305 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:48.509305 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:12:48.509305 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:12:48.509305 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:12:48.511015 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:12:48.511015 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:48.511015 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:48.511015 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:48.511015 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:48.511015 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:48.511015 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:48.511015 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:48.511015 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:48.511015 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:48.511015 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:48.511015 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:48.511015 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:48.511015 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:48.511015 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:48.511015 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:48.511015 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:48.513491 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:12:48.513491 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:12:48.513491 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:12:48.513491 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:12:48.513491 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:12:48.513491 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:12:48.513491 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:12:48.513491 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:12:48.518134 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 273/564] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:12:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 407 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:48.520077 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0f66c8-3d0a-4512-a7a3-671e2730999b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:48.579708 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:48.672190 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d28e75-8da1-40a9-8393-01740f1cb772 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:48.686117 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:48.699525 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0f66c8-3d0a-4512-a7a3-671e2730999b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:48.759327 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46273d27-5534-4da3-b576-5272538922f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:48.760910 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46273d27-5534-4da3-b576-5272538922f0 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:48.761350 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46273d27-5534-4da3-b576-5272538922f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:48.762493 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46273d27-5534-4da3-b576-5272538922f0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:48.775616 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46273d27-5534-4da3-b576-5272538922f0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:48.777587 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46273d27-5534-4da3-b576-5272538922f0 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:48.782072 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f28b4-41f2-4e9c-8293-eedeb1296342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:48.783203 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f28b4-41f2-4e9c-8293-eedeb1296342 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:48.783652 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f28b4-41f2-4e9c-8293-eedeb1296342 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:48.784097 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f28b4-41f2-4e9c-8293-eedeb1296342 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:48.785329 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81f28b4-41f2-4e9c-8293-eedeb1296342 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:48.785861 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81f28b4-41f2-4e9c-8293-eedeb1296342 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:48.788365 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81f28b4-41f2-4e9c-8293-eedeb1296342 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:48.804561 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-46273d27-5534-4da3-b576-5272538922f0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:12:48.804561 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:48.804561 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:48.804561 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:48.804561 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:48.804561 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:48.804561 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:48.804561 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:48.804561 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:48.804561 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:48.804561 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:48.804561 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:48.804561 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:12:48.804561 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:12:48.804561 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:12:48.804561 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:12:48.805685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:48.805685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:48.805685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:12:48.805685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:12:48.805685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:12:48.805685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:12:48.805685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:48.805685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:48.805685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:48.805685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:48.805685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:48.805685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:48.805685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:48.805685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:48.805685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:48.805685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:48.805685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:48.805685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:48.815729 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:48.815729 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:48.815729 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:12:48.815729 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:12:48.815729 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:12:48.815729 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:12:48.815729 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:12:48.815729 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:12:48.815729 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:12:48.815729 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:12:48.815729 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 292/565] 10.222.0.87 () {56 vars in 1093 bytes} [Tue Jun 21 18:12:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:48.816932 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 274/566] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:12:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:48.816932 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b7ce45-c13b-45ab-9e9d-976c5f0d5835 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:48.816932 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b7ce45-c13b-45ab-9e9d-976c5f0d5835 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:48.816932 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b7ce45-c13b-45ab-9e9d-976c5f0d5835 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:48.817785 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46273d27-5534-4da3-b576-5272538922f0 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:48.820547 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 275/567] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:12:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:12:48.827223 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b7ce45-c13b-45ab-9e9d-976c5f0d5835 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:49.037462 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b7ce45-c13b-45ab-9e9d-976c5f0d5835 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:49.066600 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46273d27-5534-4da3-b576-5272538922f0 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:49.095079 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:49.099527 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de738783-f766-4264-a5a4-3a600a493ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:49.101286 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de738783-f766-4264-a5a4-3a600a493ae7 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:49.101286 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de738783-f766-4264-a5a4-3a600a493ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:49.101571 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de738783-f766-4264-a5a4-3a600a493ae7 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:49.103555 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de738783-f766-4264-a5a4-3a600a493ae7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:49.104061 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de738783-f766-4264-a5a4-3a600a493ae7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:49.107156 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de738783-f766-4264-a5a4-3a600a493ae7 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:49.125170 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca434c8-32d6-4d7f-b435-d7a2433364c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:49.126055 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca434c8-32d6-4d7f-b435-d7a2433364c6 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:49.126443 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca434c8-32d6-4d7f-b435-d7a2433364c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:49.126815 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca434c8-32d6-4d7f-b435-d7a2433364c6 None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:49.127727 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca434c8-32d6-4d7f-b435-d7a2433364c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:49.128130 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca434c8-32d6-4d7f-b435-d7a2433364c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:49.129794 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca434c8-32d6-4d7f-b435-d7a2433364c6 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:49.141150 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 276/568] 10.222.0.87 () {56 vars in 1133 bytes} [Tue Jun 21 18:12:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:49.142584 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de738783-f766-4264-a5a4-3a600a493ae7 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:49.144658 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 293/569] 10.222.0.87 () {56 vars in 1148 bytes} [Tue Jun 21 18:12:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:49.147394 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288abb94-cc98-4882-a036-c8b410bd907c None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:49.148112 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288abb94-cc98-4882-a036-c8b410bd907c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:49.148539 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288abb94-cc98-4882-a036-c8b410bd907c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:49.168193 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-288abb94-cc98-4882-a036-c8b410bd907c None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:12:49.169528 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-288abb94-cc98-4882-a036-c8b410bd907c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:12:49.173340 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:49.254961 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 294/570] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:12:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:12:49.256368 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d97853-11a5-4435-8a21-1971aa0d3024 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:49.256665 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d97853-11a5-4435-8a21-1971aa0d3024 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:49.256957 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d97853-11a5-4435-8a21-1971aa0d3024 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:49.258953 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 295/571] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:12:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:12:49.259908 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d97853-11a5-4435-8a21-1971aa0d3024 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:49.325742 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de738783-f766-4264-a5a4-3a600a493ae7 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:49.380271 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396a8e54-1162-482b-a905-b200c465e99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:49.381102 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a8e54-1162-482b-a905-b200c465e99d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:49.381392 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a8e54-1162-482b-a905-b200c465e99d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:49.381705 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a8e54-1162-482b-a905-b200c465e99d None admin] PATH_INFO: `/v3/users/glance` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:49.393861 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396a8e54-1162-482b-a905-b200c465e99d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:49.396200 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396a8e54-1162-482b-a905-b200c465e99d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:49.413022 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d97853-11a5-4435-8a21-1971aa0d3024 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:49.416295 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:49.424161 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-396a8e54-1162-482b-a905-b200c465e99d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 21 18:12:49.424161 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:49.424161 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:49.424161 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:49.424161 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:49.424161 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:49.424161 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:49.424161 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:49.424161 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:49.424161 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:49.424161 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:49.424161 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:49.424161 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:12:49.424161 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:12:49.424161 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:12:49.424161 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:12:49.426231 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:49.426231 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:49.426231 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:12:49.426231 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:12:49.426231 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:12:49.426231 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:12:49.426231 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:49.426231 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:49.426231 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:49.426231 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:49.426231 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:49.426231 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:49.426231 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:49.426231 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:49.426231 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:49.426231 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:49.426231 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:49.426231 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:49.428124 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:49.428124 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:49.428124 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:12:49.428124 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:12:49.428124 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:12:49.428124 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:12:49.428124 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:12:49.428124 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:12:49.428124 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 21 18:12:49.428124 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:12:49.428124 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 277/572] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:12:49 2022] GET /identity/v3/users/glance => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:49.428124 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396a8e54-1162-482b-a905-b200c465e99d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:49.463204 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c7def7-5058-4c25-8f5c-6fd14fecb0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:49.464007 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c7def7-5058-4c25-8f5c-6fd14fecb0a9 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:49.464287 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c7def7-5058-4c25-8f5c-6fd14fecb0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:49.464559 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c7def7-5058-4c25-8f5c-6fd14fecb0a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:49.478790 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c7def7-5058-4c25-8f5c-6fd14fecb0a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:49.480674 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c7def7-5058-4c25-8f5c-6fd14fecb0a9 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:49.514694 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-16c7def7-5058-4c25-8f5c-6fd14fecb0a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:49.514694 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:49.514694 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:49.514694 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:49.514694 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:49.514694 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:49.514694 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:49.514694 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:49.514694 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:49.514694 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:49.514694 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:49.514694 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:49.514694 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:12:49.514694 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:12:49.514694 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:12:49.515698 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:12:49.515698 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:49.515698 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:49.515698 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:49.515698 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:49.515698 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:49.515698 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:49.515698 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:49.515698 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:49.515698 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:49.515698 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:49.515698 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:49.515698 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:49.515698 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:49.515698 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:49.515698 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:49.515698 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:49.516617 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:12:49.516617 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:12:49.516617 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:49.516617 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:12:49.517256 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 296/573] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:12:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:49.518865 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac6fe3-42d0-43fa-9776-6b7248c45e97 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:49.519258 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac6fe3-42d0-43fa-9776-6b7248c45e97 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:49.519542 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac6fe3-42d0-43fa-9776-6b7248c45e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:49.539006 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-f4ac6fe3-42d0-43fa-9776-6b7248c45e97 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:12:49.540240 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f4ac6fe3-42d0-43fa-9776-6b7248c45e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:12:49.553358 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:49.596139 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396a8e54-1162-482b-a905-b200c465e99d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:49.632623 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 297/574] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:12:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:12:49.633922 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac6fe3-42d0-43fa-9776-6b7248c45e97 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:49.651843 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d83d6-b933-446e-9e45-2cc0c500305c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:49.653196 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d83d6-b933-446e-9e45-2cc0c500305c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:49.653772 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d83d6-b933-446e-9e45-2cc0c500305c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:49.654245 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d83d6-b933-446e-9e45-2cc0c500305c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:49.655287 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58d83d6-b933-446e-9e45-2cc0c500305c None admin] RBAC: Adding query filter params (role.id=b9dbf55e6f2947158c1d946f6ee9f907, scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb, user.id=515b3571045e4fe29565307c8ecc10b2) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:49.655725 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58d83d6-b933-446e-9e45-2cc0c500305c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:49.657310 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58d83d6-b933-446e-9e45-2cc0c500305c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:49.677250 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 278/575] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:12:49 2022] GET /identity/v3/role_assignments?user.id=515b3571045e4fe29565307c8ecc10b2&scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb&role.id=b9dbf55e6f2947158c1d946f6ee9f907 => generated 259 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:49.677250 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d83d6-b933-446e-9e45-2cc0c500305c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:49.802260 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:49.834277 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac6fe3-42d0-43fa-9776-6b7248c45e97 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:49.859707 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d83d6-b933-446e-9e45-2cc0c500305c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:49.897604 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4495b5-111d-4182-a82e-61543f4daccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:49.898894 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4495b5-111d-4182-a82e-61543f4daccc None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:49.899354 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4495b5-111d-4182-a82e-61543f4daccc None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:49.899816 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4495b5-111d-4182-a82e-61543f4daccc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:49.922723 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4495b5-111d-4182-a82e-61543f4daccc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:12:49.928585 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c926f9-4d0e-4561-9769-a6ee56d63fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:49.929453 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c926f9-4d0e-4561-9769-a6ee56d63fcd None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:49.929901 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c926f9-4d0e-4561-9769-a6ee56d63fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:49.930360 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c926f9-4d0e-4561-9769-a6ee56d63fcd None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:49.932270 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c926f9-4d0e-4561-9769-a6ee56d63fcd None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:49.932871 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c926f9-4d0e-4561-9769-a6ee56d63fcd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:49.935865 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c926f9-4d0e-4561-9769-a6ee56d63fcd None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:49.938626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-dd4495b5-111d-4182-a82e-61543f4daccc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:12:49.938626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:49.938626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:49.938626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:49.938626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:49.938626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:49.938626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:49.938626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:49.938626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:49.938626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:49.938626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:49.938626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:49.938626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:12:49.938626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:12:49.938626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:12:49.938626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:12:49.940142 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:12:49.940142 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:12:49.940142 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:12:49.940142 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:12:49.940142 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:49.940142 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:49.940142 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:49.940142 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:49.940142 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:49.940142 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:49.940142 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:49.940142 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:49.940142 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:49.940142 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:49.940142 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:49.940142 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:49.941417 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:49.941417 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:49.941417 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:49.941417 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:49.941417 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:12:49.941417 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:12:49.941417 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:12:49.941417 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:12:49.941417 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:12:49.941417 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:12:49.941417 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:12:49.941417 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:12:49.941417 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 298/576] 10.222.0.87 () {56 vars in 1102 bytes} [Tue Jun 21 18:12:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:49.942483 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4495b5-111d-4182-a82e-61543f4daccc None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:49.964039 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:49.983557 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 279/577] 10.222.0.87 () {56 vars in 1137 bytes} [Tue Jun 21 18:12:49 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:49.985231 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c926f9-4d0e-4561-9769-a6ee56d63fcd None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:50.195514 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c926f9-4d0e-4561-9769-a6ee56d63fcd None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:50.197712 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4495b5-111d-4182-a82e-61543f4daccc None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:50.267006 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98330623-7a28-40e3-8619-cfabe7623b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:50.269591 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98330623-7a28-40e3-8619-cfabe7623b0d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:50.269591 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98330623-7a28-40e3-8619-cfabe7623b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:50.269591 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98330623-7a28-40e3-8619-cfabe7623b0d None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:50.270674 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98330623-7a28-40e3-8619-cfabe7623b0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:50.271203 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98330623-7a28-40e3-8619-cfabe7623b0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:50.276453 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98330623-7a28-40e3-8619-cfabe7623b0d None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:50.280928 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dbd518-2668-4b19-b890-387c19a29d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:50.282143 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dbd518-2668-4b19-b890-387c19a29d35 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:50.282565 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dbd518-2668-4b19-b890-387c19a29d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:50.282974 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dbd518-2668-4b19-b890-387c19a29d35 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:50.304806 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 299/578] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:12:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:50.304806 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98330623-7a28-40e3-8619-cfabe7623b0d None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:50.306912 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dbd518-2668-4b19-b890-387c19a29d35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:12:50.317990 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-37dbd518-2668-4b19-b890-387c19a29d35 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:12:50.317990 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:50.317990 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:50.317990 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:50.317990 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:50.317990 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:50.317990 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:50.317990 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:50.317990 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:50.317990 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:50.317990 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:50.317990 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:50.317990 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:12:50.317990 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:12:50.317990 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:12:50.317990 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:12:50.326946 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:12:50.326946 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:12:50.326946 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:12:50.326946 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:12:50.326946 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:50.326946 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:50.326946 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:50.326946 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:50.326946 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:50.326946 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:50.326946 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:50.326946 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:50.326946 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:50.326946 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:50.326946 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:50.326946 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:50.327875 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:50.327875 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:50.327875 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:50.327875 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:50.327875 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:12:50.327875 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:12:50.327875 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:12:50.327875 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:12:50.327875 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:12:50.327875 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:12:50.327875 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:12:50.327875 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:12:50.327875 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 280/579] 10.222.0.87 () {56 vars in 1102 bytes} [Tue Jun 21 18:12:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:50.327875 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dbd518-2668-4b19-b890-387c19a29d35 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:50.330425 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:50.445715 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:50.542029 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dbd518-2668-4b19-b890-387c19a29d35 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:50.543184 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98330623-7a28-40e3-8619-cfabe7623b0d None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:50.612529 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e3ef4e-86c2-4b27-8f66-0ffbfa2f31b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:50.613914 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3ef4e-86c2-4b27-8f66-0ffbfa2f31b3 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:50.614451 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3ef4e-86c2-4b27-8f66-0ffbfa2f31b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:50.615012 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3ef4e-86c2-4b27-8f66-0ffbfa2f31b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:50.618513 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5932a27-d6f7-4a4e-b9e8-d454a1b26e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:50.619997 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5932a27-d6f7-4a4e-b9e8-d454a1b26e05 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:50.620748 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5932a27-d6f7-4a4e-b9e8-d454a1b26e05 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:50.621311 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5932a27-d6f7-4a4e-b9e8-d454a1b26e05 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:50.623042 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5932a27-d6f7-4a4e-b9e8-d454a1b26e05 None admin] RBAC: Adding query filter params (name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:50.623588 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5932a27-d6f7-4a4e-b9e8-d454a1b26e05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:50.626233 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5932a27-d6f7-4a4e-b9e8-d454a1b26e05 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:50.638113 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e3ef4e-86c2-4b27-8f66-0ffbfa2f31b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:50.641165 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e3ef4e-86c2-4b27-8f66-0ffbfa2f31b3 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:50.644753 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 300/580] 10.222.0.87 () {56 vars in 1103 bytes} [Tue Jun 21 18:12:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:50.646685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5932a27-d6f7-4a4e-b9e8-d454a1b26e05 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:50.675463 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-14e3ef4e-86c2-4b27-8f66-0ffbfa2f31b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:50.675463 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:50.675463 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:50.675463 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:50.675463 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:50.675463 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:50.675463 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:50.675463 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:50.675463 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:50.675463 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:50.675463 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:50.675463 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:50.675463 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:12:50.675463 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:12:50.675463 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:12:50.677086 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:12:50.677086 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:50.677086 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:50.677086 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:50.677086 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:50.677086 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:50.677086 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:50.677086 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:50.677086 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:50.677086 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:50.677086 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:50.677086 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:50.677086 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:50.677086 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:50.677086 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:50.677086 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:50.677086 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:50.679007 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:12:50.679007 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:12:50.679007 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:50.679007 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:12:50.679007 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 281/581] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:12:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:50.679530 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e3ef4e-86c2-4b27-8f66-0ffbfa2f31b3 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:50.794747 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:50.880047 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e3ef4e-86c2-4b27-8f66-0ffbfa2f31b3 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:50.881025 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5932a27-d6f7-4a4e-b9e8-d454a1b26e05 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:50.901993 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:50.940438 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ce087-3a65-4add-ace8-443ce18b5793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:50.941523 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ce087-3a65-4add-ace8-443ce18b5793 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:50.941844 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ce087-3a65-4add-ace8-443ce18b5793 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:50.942208 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ce087-3a65-4add-ace8-443ce18b5793 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:50.952047 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62760148-a033-4a77-9bf7-1971ec60f084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:50.952878 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62760148-a033-4a77-9bf7-1971ec60f084 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:50.953185 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62760148-a033-4a77-9bf7-1971ec60f084 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:50.953621 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62760148-a033-4a77-9bf7-1971ec60f084 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:50.954806 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8ce087-3a65-4add-ace8-443ce18b5793 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:50.955010 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62760148-a033-4a77-9bf7-1971ec60f084 None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:50.955336 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62760148-a033-4a77-9bf7-1971ec60f084 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:50.957479 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8ce087-3a65-4add-ace8-443ce18b5793 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:50.957867 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62760148-a033-4a77-9bf7-1971ec60f084 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:50.984569 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 282/582] 10.222.0.87 () {56 vars in 1103 bytes} [Tue Jun 21 18:12:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:50.984569 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62760148-a033-4a77-9bf7-1971ec60f084 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:50.987958 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-3f8ce087-3a65-4add-ace8-443ce18b5793 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:12:50.987958 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:50.987958 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:50.987958 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:50.987958 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:50.987958 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:50.987958 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:50.987958 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:50.987958 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:50.987958 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:50.987958 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:50.987958 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:50.987958 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:12:50.987958 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:12:50.987958 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:12:50.989243 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:12:50.989243 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:50.989243 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:50.989243 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:50.989243 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:50.989243 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:50.989243 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:50.989243 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:50.989243 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:50.989243 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:50.989243 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:50.989243 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:50.989243 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:50.989243 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:50.989243 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:50.989243 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:50.989243 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:50.990156 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:12:50.990156 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:12:50.990156 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:12:50.990156 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:12:50.990156 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:12:50.990156 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:12:50.990156 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:12:50.990156 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:12:50.990156 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 301/583] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:12:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:50.991336 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ce087-3a65-4add-ace8-443ce18b5793 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:51.199895 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62760148-a033-4a77-9bf7-1971ec60f084 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:51.205771 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ce087-3a65-4add-ace8-443ce18b5793 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:51.253962 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:51.267481 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738aaa62-d585-4fda-9094-8dc1ea1b6a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:51.268324 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738aaa62-d585-4fda-9094-8dc1ea1b6a57 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:51.268807 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738aaa62-d585-4fda-9094-8dc1ea1b6a57 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:51.269131 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738aaa62-d585-4fda-9094-8dc1ea1b6a57 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:51.269955 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738aaa62-d585-4fda-9094-8dc1ea1b6a57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:51.270268 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738aaa62-d585-4fda-9094-8dc1ea1b6a57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:51.271713 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738aaa62-d585-4fda-9094-8dc1ea1b6a57 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:51.279299 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1420f8e-e52f-4e87-891c-83c8a966a7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:51.281213 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1420f8e-e52f-4e87-891c-83c8a966a7ce None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:51.283080 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1420f8e-e52f-4e87-891c-83c8a966a7ce None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:51.284045 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1420f8e-e52f-4e87-891c-83c8a966a7ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:51.298961 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 302/584] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:12:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:51.298961 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738aaa62-d585-4fda-9094-8dc1ea1b6a57 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:51.317090 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1420f8e-e52f-4e87-891c-83c8a966a7ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:51.321256 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1420f8e-e52f-4e87-891c-83c8a966a7ce None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:51.356525 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-f1420f8e-e52f-4e87-891c-83c8a966a7ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:51.356525 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:51.356525 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:51.356525 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:51.356525 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:51.356525 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:51.356525 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:51.356525 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:51.356525 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:51.356525 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:51.356525 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:51.356525 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:51.356525 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:12:51.356525 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:12:51.356525 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:12:51.358402 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:12:51.358402 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:51.358402 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:51.358402 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:51.358402 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:51.358402 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:51.358402 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:51.358402 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:51.358402 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:51.358402 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:51.358402 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:51.358402 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:51.358402 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:51.358402 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:51.358402 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:51.358402 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:51.358402 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:51.360173 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:12:51.360173 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:12:51.360173 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:51.360173 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:12:51.365796 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 283/585] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:12:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 392 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:51.367126 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1420f8e-e52f-4e87-891c-83c8a966a7ce None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:51.416847 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:51.534644 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738aaa62-d585-4fda-9094-8dc1ea1b6a57 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:51.593192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1420f8e-e52f-4e87-891c-83c8a966a7ce None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:51.604675 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d0af33-939d-435b-82ac-97181f54c6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:51.605878 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d0af33-939d-435b-82ac-97181f54c6e5 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:51.606356 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d0af33-939d-435b-82ac-97181f54c6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:51.606846 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d0af33-939d-435b-82ac-97181f54c6e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:51.624812 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d0af33-939d-435b-82ac-97181f54c6e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:51.626582 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d0af33-939d-435b-82ac-97181f54c6e5 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:51.661516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-f7d0af33-939d-435b-82ac-97181f54c6e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:51.661516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:51.661516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:51.661516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:51.661516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:51.661516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:51.661516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:51.661516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:51.661516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:51.661516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:51.661516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:51.661516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:51.661516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:12:51.661516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:12:51.661516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:12:51.663390 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:12:51.663390 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:51.663390 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:51.663390 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:51.663390 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:51.663390 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:51.663390 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:51.663390 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:51.663390 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:51.663390 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:51.663390 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:51.663390 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:51.663390 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:51.663390 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:51.663390 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:51.663390 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:51.663390 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:51.665171 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:12:51.665171 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:12:51.665171 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:51.665171 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:12:51.665600 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f881b-f871-490f-983c-0c821f0285b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:51.665600 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f881b-f871-490f-983c-0c821f0285b8 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:51.665600 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f881b-f871-490f-983c-0c821f0285b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:51.674032 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 303/586] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:12:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:51.674182 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f881b-f871-490f-983c-0c821f0285b8 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:51.674352 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d0af33-939d-435b-82ac-97181f54c6e5 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:51.674486 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0f881b-f871-490f-983c-0c821f0285b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:51.674486 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0f881b-f871-490f-983c-0c821f0285b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:51.674486 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0f881b-f871-490f-983c-0c821f0285b8 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:51.699494 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 284/587] 10.222.0.87 () {56 vars in 1148 bytes} [Tue Jun 21 18:12:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:51.700656 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f881b-f871-490f-983c-0c821f0285b8 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:51.766071 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:51.876549 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:51.929227 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f881b-f871-490f-983c-0c821f0285b8 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:51.938691 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d0af33-939d-435b-82ac-97181f54c6e5 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:52.018899 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb28a13-77f7-4c9b-bf57-54a253d5317e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:52.020638 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb28a13-77f7-4c9b-bf57-54a253d5317e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:52.021164 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb28a13-77f7-4c9b-bf57-54a253d5317e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:52.021683 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb28a13-77f7-4c9b-bf57-54a253d5317e None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:52.023137 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb28a13-77f7-4c9b-bf57-54a253d5317e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:52.023901 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb28a13-77f7-4c9b-bf57-54a253d5317e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:52.026478 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb28a13-77f7-4c9b-bf57-54a253d5317e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:52.036725 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576f9e7c-c77d-406f-90bd-894951cd1e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:52.038183 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f9e7c-c77d-406f-90bd-894951cd1e49 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:52.038676 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f9e7c-c77d-406f-90bd-894951cd1e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:52.039180 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f9e7c-c77d-406f-90bd-894951cd1e49 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:52.052298 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576f9e7c-c77d-406f-90bd-894951cd1e49 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:52.054142 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576f9e7c-c77d-406f-90bd-894951cd1e49 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:52.077063 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 285/588] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:12:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:52.077063 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb28a13-77f7-4c9b-bf57-54a253d5317e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:52.078204 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-576f9e7c-c77d-406f-90bd-894951cd1e49 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:12:52.078204 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:52.078204 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:52.078204 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:52.078204 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:52.078204 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:52.078204 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:52.078204 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:52.078204 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:52.078204 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:52.078204 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:52.078204 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:52.078204 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:12:52.078204 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:12:52.078204 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:12:52.079950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:12:52.079950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:52.079950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:52.079950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:52.079950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:52.079950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:52.079950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:52.079950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:52.079950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:52.079950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:52.079950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:52.079950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:52.079950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:52.079950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:52.079950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:52.079950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:52.079950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:52.082863 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:12:52.082863 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:12:52.082863 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:12:52.082863 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:12:52.082863 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:12:52.082863 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:12:52.082863 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:12:52.082863 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:12:52.082863 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 304/589] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:12:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 415 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:52.082863 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576f9e7c-c77d-406f-90bd-894951cd1e49 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:52.320184 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:52.327518 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576f9e7c-c77d-406f-90bd-894951cd1e49 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:52.343198 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb28a13-77f7-4c9b-bf57-54a253d5317e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:52.401050 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86eabf-344a-411c-bbc5-ee6fa0c90b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:52.403022 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86eabf-344a-411c-bbc5-ee6fa0c90b01 None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:52.403580 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86eabf-344a-411c-bbc5-ee6fa0c90b01 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:52.404070 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86eabf-344a-411c-bbc5-ee6fa0c90b01 None admin] PATH_INFO: `/v3/projects/5f30c3ddb5e44e9299ff228fe94c27fb/users/a1b3fa67905c4027b3656ac36e2b76c9/roles/b9dbf55e6f2947158c1d946f6ee9f907` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:52.406855 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd929884-f014-422a-9e5c-3c34a7624988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:52.407569 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd929884-f014-422a-9e5c-3c34a7624988 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:52.407909 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd929884-f014-422a-9e5c-3c34a7624988 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:52.408192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd929884-f014-422a-9e5c-3c34a7624988 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:52.408953 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd929884-f014-422a-9e5c-3c34a7624988 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:52.409265 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd929884-f014-422a-9e5c-3c34a7624988 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:52.410998 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd929884-f014-422a-9e5c-3c34a7624988 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:52.440656 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 286/590] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:12:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:52.442390 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd929884-f014-422a-9e5c-3c34a7624988 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:52.447752 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:52.451626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a86eabf-344a-411c-bbc5-ee6fa0c90b01 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f30c3ddb5e44e9299ff228fe94c27fb, user_id=a1b3fa67905c4027b3656ac36e2b76c9, role_id=b9dbf55e6f2947158c1d946f6ee9f907)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:52.452601 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a86eabf-344a-411c-bbc5-ee6fa0c90b01 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:52.475622 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 305/591] 10.222.0.87 () {58 vars in 1439 bytes} [Tue Jun 21 18:12:52 2022] PUT /identity/v3/projects/5f30c3ddb5e44e9299ff228fe94c27fb/users/a1b3fa67905c4027b3656ac36e2b76c9/roles/b9dbf55e6f2947158c1d946f6ee9f907 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:12:52.476971 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86eabf-344a-411c-bbc5-ee6fa0c90b01 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:52.700122 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd929884-f014-422a-9e5c-3c34a7624988 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:52.723809 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86eabf-344a-411c-bbc5-ee6fa0c90b01 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:52.779568 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812676d-afa3-490d-a066-08c43297fa42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:52.781748 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812676d-afa3-490d-a066-08c43297fa42 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:52.782561 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812676d-afa3-490d-a066-08c43297fa42 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:52.783529 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812676d-afa3-490d-a066-08c43297fa42 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:52.786812 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a006c4c-2b79-4ba0-9bdb-b65f16b8e46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:52.787979 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a006c4c-2b79-4ba0-9bdb-b65f16b8e46f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:52.788483 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a006c4c-2b79-4ba0-9bdb-b65f16b8e46f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:52.789595 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a006c4c-2b79-4ba0-9bdb-b65f16b8e46f None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:52.790165 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a006c4c-2b79-4ba0-9bdb-b65f16b8e46f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:52.790487 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a006c4c-2b79-4ba0-9bdb-b65f16b8e46f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:52.792933 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a006c4c-2b79-4ba0-9bdb-b65f16b8e46f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:52.800888 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b812676d-afa3-490d-a066-08c43297fa42 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:52.803181 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b812676d-afa3-490d-a066-08c43297fa42 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:52.825836 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 306/592] 10.222.0.87 () {56 vars in 1148 bytes} [Tue Jun 21 18:12:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:52.825836 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a006c4c-2b79-4ba0-9bdb-b65f16b8e46f None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:52.836527 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-b812676d-afa3-490d-a066-08c43297fa42 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 21 18:12:52.836527 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:52.836527 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:52.836527 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:52.836527 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:52.836527 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:52.836527 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:52.836527 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:52.836527 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:52.836527 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:52.836527 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:52.836527 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:52.836527 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:12:52.836527 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:12:52.836527 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:12:52.836527 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:12:52.837689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:52.837689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:52.837689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:12:52.837689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:12:52.837689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:12:52.837689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:12:52.837689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:52.837689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:52.837689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:52.837689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:52.837689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:52.837689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:52.837689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:52.837689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:52.837689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:52.837689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:52.837689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:52.837689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:52.838650 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:52.838650 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:52.838650 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:12:52.838650 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:12:52.838650 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:12:52.838650 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:12:52.838650 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:12:52.838650 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:12:52.838650 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 21 18:12:52.838650 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:12:52.839536 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 287/593] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:12:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 398 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:52.845698 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcea039-25f1-4231-a4b4-26c873b4c348 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:52.845698 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcea039-25f1-4231-a4b4-26c873b4c348 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:52.845698 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcea039-25f1-4231-a4b4-26c873b4c348 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:52.847098 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 288/594] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:12:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:12:52.853680 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcea039-25f1-4231-a4b4-26c873b4c348 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:52.860077 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:53.028759 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:53.075299 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a006c4c-2b79-4ba0-9bdb-b65f16b8e46f None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:53.115231 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcea039-25f1-4231-a4b4-26c873b4c348 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:53.158734 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59bd2c4-6a94-4e41-9daf-163214062963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:53.160671 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59bd2c4-6a94-4e41-9daf-163214062963 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:53.161533 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59bd2c4-6a94-4e41-9daf-163214062963 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:53.162164 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59bd2c4-6a94-4e41-9daf-163214062963 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:53.178993 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc63bb5-f9b5-4a8b-9e55-b39fee333d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:53.179934 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc63bb5-f9b5-4a8b-9e55-b39fee333d96 None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:53.180218 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc63bb5-f9b5-4a8b-9e55-b39fee333d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:53.180680 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc63bb5-f9b5-4a8b-9e55-b39fee333d96 None admin] PATH_INFO: `/v3/projects/5f30c3ddb5e44e9299ff228fe94c27fb/users/e7da4bf6f3be47b4aa25c6240cb4be03/roles/b9dbf55e6f2947158c1d946f6ee9f907` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:53.186036 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59bd2c4-6a94-4e41-9daf-163214062963 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:53.188957 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59bd2c4-6a94-4e41-9daf-163214062963 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:53.202856 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc63bb5-f9b5-4a8b-9e55-b39fee333d96 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f30c3ddb5e44e9299ff228fe94c27fb, user_id=e7da4bf6f3be47b4aa25c6240cb4be03, role_id=b9dbf55e6f2947158c1d946f6ee9f907)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:53.204428 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc63bb5-f9b5-4a8b-9e55-b39fee333d96 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:53.219485 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-c59bd2c4-6a94-4e41-9daf-163214062963 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 21 18:12:53.219485 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:53.219485 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:53.219485 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:53.219485 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:53.219485 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:53.219485 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:53.219485 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:53.219485 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:53.219485 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:53.219485 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:53.219485 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:53.219485 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:12:53.219485 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:12:53.219485 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:12:53.219485 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:12:53.221575 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:53.221575 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:53.221575 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:12:53.221575 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:12:53.221575 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:12:53.221575 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:12:53.221575 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:53.221575 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:53.221575 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:53.221575 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:53.221575 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:53.221575 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:53.221575 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:53.221575 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:53.221575 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:53.221575 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:53.221575 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:53.221575 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:53.223611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:53.223611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:53.223611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:12:53.223611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:12:53.223611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:12:53.223611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:12:53.223611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:12:53.223611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:12:53.223611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 21 18:12:53.223611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:12:53.227117 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 289/595] 10.222.0.87 () {58 vars in 1439 bytes} [Tue Jun 21 18:12:52 2022] PUT /identity/v3/projects/5f30c3ddb5e44e9299ff228fe94c27fb/users/e7da4bf6f3be47b4aa25c6240cb4be03/roles/b9dbf55e6f2947158c1d946f6ee9f907 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:12:53.228228 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 307/596] 10.222.0.87 () {56 vars in 1102 bytes} [Tue Jun 21 18:12:52 2022] GET /identity/v3/users/neutron => generated 85 bytes in 406 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:53.231164 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971880df-6c2a-4ab5-a64f-330c25994ae2 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:53.231405 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc63bb5-f9b5-4a8b-9e55-b39fee333d96 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:53.232277 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971880df-6c2a-4ab5-a64f-330c25994ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:53.233086 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971880df-6c2a-4ab5-a64f-330c25994ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:53.265771 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-971880df-6c2a-4ab5-a64f-330c25994ae2 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:12:53.268649 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-971880df-6c2a-4ab5-a64f-330c25994ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:12:53.428354 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:53.440685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 308/597] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:12:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:12:53.442732 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971880df-6c2a-4ab5-a64f-330c25994ae2 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:53.464531 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc63bb5-f9b5-4a8b-9e55-b39fee333d96 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:53.496957 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:53.550748 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1656fb5d-df6c-4c24-aa6d-701c7276b9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:53.552582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656fb5d-df6c-4c24-aa6d-701c7276b9b6 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:53.553451 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656fb5d-df6c-4c24-aa6d-701c7276b9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:53.554260 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656fb5d-df6c-4c24-aa6d-701c7276b9b6 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:53.556448 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1656fb5d-df6c-4c24-aa6d-701c7276b9b6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:53.557386 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1656fb5d-df6c-4c24-aa6d-701c7276b9b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:53.561159 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1656fb5d-df6c-4c24-aa6d-701c7276b9b6 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:53.599409 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 290/598] 10.222.0.87 () {56 vars in 1137 bytes} [Tue Jun 21 18:12:53 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:53.601161 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1656fb5d-df6c-4c24-aa6d-701c7276b9b6 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:53.662474 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971880df-6c2a-4ab5-a64f-330c25994ae2 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:53.733567 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea26fb-2192-4469-ac51-71dcc355f8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:53.734915 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea26fb-2192-4469-ac51-71dcc355f8c4 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:53.735416 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea26fb-2192-4469-ac51-71dcc355f8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:53.735915 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea26fb-2192-4469-ac51-71dcc355f8c4 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:53.737952 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ea26fb-2192-4469-ac51-71dcc355f8c4 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:53.738586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ea26fb-2192-4469-ac51-71dcc355f8c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:53.741335 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ea26fb-2192-4469-ac51-71dcc355f8c4 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:53.787320 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 309/599] 10.222.0.87 () {56 vars in 1139 bytes} [Tue Jun 21 18:12:53 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:53.789054 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea26fb-2192-4469-ac51-71dcc355f8c4 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:53.819640 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1656fb5d-df6c-4c24-aa6d-701c7276b9b6 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:53.910164 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2923266a-2d3b-4131-a2d3-565dfa2387fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:53.912251 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2923266a-2d3b-4131-a2d3-565dfa2387fe None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:53.913523 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2923266a-2d3b-4131-a2d3-565dfa2387fe None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:53.914383 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2923266a-2d3b-4131-a2d3-565dfa2387fe None admin] PATH_INFO: `/v3/roles/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:53.932993 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:53.943958 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2923266a-2d3b-4131-a2d3-565dfa2387fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:12:53.957206 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-2923266a-2d3b-4131-a2d3-565dfa2387fe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:12:53.957206 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:53.957206 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:53.957206 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:53.957206 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:53.957206 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:53.957206 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:53.957206 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:53.957206 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:53.957206 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:53.957206 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:53.957206 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:53.957206 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:12:53.957206 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:12:53.957206 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:12:53.957206 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:12:53.958284 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:12:53.958284 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:12:53.958284 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:12:53.958284 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:12:53.958284 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:53.958284 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:53.958284 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:53.958284 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:53.958284 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:53.958284 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:53.958284 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:53.958284 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:53.958284 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:53.958284 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:53.958284 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:53.958284 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:53.959906 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:53.959906 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:53.959906 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:53.959906 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:53.959906 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:12:53.959906 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:12:53.959906 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:12:53.959906 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:12:53.959906 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:12:53.959906 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:12:53.959906 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:12:53.959906 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:12:53.973199 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 291/600] 10.222.0.87 () {56 vars in 1102 bytes} [Tue Jun 21 18:12:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:53.974651 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2923266a-2d3b-4131-a2d3-565dfa2387fe None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:54.023512 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea26fb-2192-4469-ac51-71dcc355f8c4 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:54.040132 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:54.097507 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5816591-c5f9-40ad-b048-0b93b5b6fec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:54.099872 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5816591-c5f9-40ad-b048-0b93b5b6fec7 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:54.102126 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5816591-c5f9-40ad-b048-0b93b5b6fec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:54.103067 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5816591-c5f9-40ad-b048-0b93b5b6fec7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:54.125375 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5816591-c5f9-40ad-b048-0b93b5b6fec7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:54.129970 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5816591-c5f9-40ad-b048-0b93b5b6fec7 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:54.163616 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-d5816591-c5f9-40ad-b048-0b93b5b6fec7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:54.163616 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:54.163616 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:54.163616 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:54.163616 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:54.163616 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:54.163616 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:54.163616 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:54.163616 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:54.163616 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:54.163616 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:54.163616 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:54.163616 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:12:54.163616 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:12:54.163616 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:12:54.164657 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:12:54.164657 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:54.164657 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:54.164657 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:54.164657 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:54.164657 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:54.164657 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:54.164657 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:54.164657 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:54.164657 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:54.164657 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:54.164657 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:54.164657 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:54.164657 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:54.164657 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:54.164657 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:54.164657 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:54.165569 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:12:54.165569 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:12:54.165569 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:54.165569 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:12:54.170162 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 310/601] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:12:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:54.171901 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5816591-c5f9-40ad-b048-0b93b5b6fec7 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:54.216545 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2923266a-2d3b-4131-a2d3-565dfa2387fe None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:54.297686 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad8d182-eb32-4d87-933a-784ecca0981a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:54.299362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8d182-eb32-4d87-933a-784ecca0981a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:54.299600 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8d182-eb32-4d87-933a-784ecca0981a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:54.300306 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8d182-eb32-4d87-933a-784ecca0981a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:54.321945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad8d182-eb32-4d87-933a-784ecca0981a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:54.324738 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad8d182-eb32-4d87-933a-784ecca0981a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:54.336248 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5816591-c5f9-40ad-b048-0b93b5b6fec7 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:54.355005 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:54.368603 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-cad8d182-eb32-4d87-933a-784ecca0981a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:54.368603 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:54.368603 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:54.368603 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:54.368603 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:54.368603 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:54.368603 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:54.368603 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:54.368603 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:54.368603 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:54.368603 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:54.368603 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:54.368603 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:12:54.368603 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:12:54.368603 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:12:54.370424 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:12:54.370424 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:54.370424 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:54.370424 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:54.370424 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:54.370424 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:54.370424 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:54.370424 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:54.370424 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:54.370424 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:54.370424 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:54.370424 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:54.370424 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:54.370424 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:54.370424 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:54.370424 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:54.370424 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:54.372287 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:12:54.372287 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:12:54.372287 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:54.372287 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:12:54.382219 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 292/602] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:12:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 408 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:54.383857 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad8d182-eb32-4d87-933a-784ecca0981a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:54.421431 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee0d71-ee6f-4ccb-9449-2fe125843622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:54.423812 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee0d71-ee6f-4ccb-9449-2fe125843622 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:54.424817 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee0d71-ee6f-4ccb-9449-2fe125843622 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:54.425789 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee0d71-ee6f-4ccb-9449-2fe125843622 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:54.427823 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ee0d71-ee6f-4ccb-9449-2fe125843622 None admin] RBAC: Adding query filter params (name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:54.429297 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ee0d71-ee6f-4ccb-9449-2fe125843622 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:54.433499 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ee0d71-ee6f-4ccb-9449-2fe125843622 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:54.449551 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 311/603] 10.222.0.87 () {56 vars in 1103 bytes} [Tue Jun 21 18:12:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:54.451329 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee0d71-ee6f-4ccb-9449-2fe125843622 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:54.524986 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:54.558829 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad8d182-eb32-4d87-933a-784ecca0981a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:54.622806 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee0d71-ee6f-4ccb-9449-2fe125843622 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:54.625387 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe6cec-d6f7-4304-80a1-a4f9e6973038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:54.626648 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe6cec-d6f7-4304-80a1-a4f9e6973038 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:54.627196 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe6cec-d6f7-4304-80a1-a4f9e6973038 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:54.627716 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe6cec-d6f7-4304-80a1-a4f9e6973038 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:54.629065 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfe6cec-d6f7-4304-80a1-a4f9e6973038 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:54.631297 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfe6cec-d6f7-4304-80a1-a4f9e6973038 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:54.633092 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfe6cec-d6f7-4304-80a1-a4f9e6973038 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:54.665807 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 293/604] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:12:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:54.667456 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe6cec-d6f7-4304-80a1-a4f9e6973038 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:54.698101 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dcdb69-f41e-413b-808a-8ed99f91bedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:54.699900 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dcdb69-f41e-413b-808a-8ed99f91bedc None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:54.700687 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dcdb69-f41e-413b-808a-8ed99f91bedc None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:54.701448 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dcdb69-f41e-413b-808a-8ed99f91bedc None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:54.703224 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dcdb69-f41e-413b-808a-8ed99f91bedc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:54.704192 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dcdb69-f41e-413b-808a-8ed99f91bedc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:54.707210 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dcdb69-f41e-413b-808a-8ed99f91bedc None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:54.733189 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 312/605] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:12:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:54.735293 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dcdb69-f41e-413b-808a-8ed99f91bedc None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:54.800087 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:54.900800 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe6cec-d6f7-4304-80a1-a4f9e6973038 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:54.947981 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dcdb69-f41e-413b-808a-8ed99f91bedc None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:54.977164 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd0b2bc-91fb-4ef6-aae1-c9d57353dbd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:54.979106 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0b2bc-91fb-4ef6-aae1-c9d57353dbd3 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:54.979106 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0b2bc-91fb-4ef6-aae1-c9d57353dbd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:54.979535 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0b2bc-91fb-4ef6-aae1-c9d57353dbd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:54.994475 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd0b2bc-91fb-4ef6-aae1-c9d57353dbd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:54.996792 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd0b2bc-91fb-4ef6-aae1-c9d57353dbd3 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:54.999498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:55.000456 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf065b89-1d21-4190-a323-d45cf3e63d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:55.002720 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf065b89-1d21-4190-a323-d45cf3e63d5d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:55.003734 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf065b89-1d21-4190-a323-d45cf3e63d5d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:55.004501 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf065b89-1d21-4190-a323-d45cf3e63d5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:55.017258 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf065b89-1d21-4190-a323-d45cf3e63d5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:55.020656 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf065b89-1d21-4190-a323-d45cf3e63d5d None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:55.021627 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-bdd0b2bc-91fb-4ef6-aae1-c9d57353dbd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:55.021627 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:55.021627 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:55.021627 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:55.021627 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:55.021627 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:55.021627 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:55.021627 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:55.021627 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:55.021627 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:55.021627 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:55.021627 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:55.021627 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:12:55.021627 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:12:55.021627 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:12:55.023301 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:12:55.023301 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:55.023301 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:55.023301 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:55.023301 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:55.023301 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:55.023301 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:55.023301 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:55.023301 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:55.023301 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:55.023301 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:55.023301 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:55.023301 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:55.023301 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:55.023301 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:55.023301 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:55.023301 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:55.025069 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:12:55.025069 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:12:55.025069 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:55.025069 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:12:55.029222 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 294/606] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:12:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:55.030662 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd0b2bc-91fb-4ef6-aae1-c9d57353dbd3 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:55.054841 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-bf065b89-1d21-4190-a323-d45cf3e63d5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:12:55.054841 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:55.054841 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:55.054841 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:55.054841 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:55.054841 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:55.054841 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:55.054841 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:55.054841 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:55.054841 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:55.054841 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:55.054841 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:55.054841 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:12:55.054841 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:12:55.054841 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:12:55.056489 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:12:55.056489 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:55.056489 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:55.056489 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:55.056489 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:55.056489 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:55.056489 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:55.056489 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:55.056489 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:55.056489 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:55.056489 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:55.056489 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:55.056489 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:55.056489 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:55.056489 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:55.056489 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:55.056489 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:55.058250 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:12:55.058250 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:12:55.058250 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:12:55.058250 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:12:55.058250 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:12:55.058250 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:12:55.058250 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:12:55.058250 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:12:55.064123 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 313/607] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:12:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:55.066578 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf065b89-1d21-4190-a323-d45cf3e63d5d None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:55.195243 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd0b2bc-91fb-4ef6-aae1-c9d57353dbd3 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:55.225885 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:55.241584 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf065b89-1d21-4190-a323-d45cf3e63d5d None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:55.251483 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f39dd7-3275-40dd-a0cb-a7eda4a71bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:55.254165 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f39dd7-3275-40dd-a0cb-a7eda4a71bcc None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:55.254905 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f39dd7-3275-40dd-a0cb-a7eda4a71bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:55.255478 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f39dd7-3275-40dd-a0cb-a7eda4a71bcc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:55.270559 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f39dd7-3275-40dd-a0cb-a7eda4a71bcc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:55.274550 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f39dd7-3275-40dd-a0cb-a7eda4a71bcc None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:55.303559 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32270a1-57cc-4340-b754-391b53b4b954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:55.304980 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32270a1-57cc-4340-b754-391b53b4b954 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:55.305525 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32270a1-57cc-4340-b754-391b53b4b954 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:55.306173 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32270a1-57cc-4340-b754-391b53b4b954 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:55.307580 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32270a1-57cc-4340-b754-391b53b4b954 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:55.308235 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32270a1-57cc-4340-b754-391b53b4b954 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:55.309590 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-79f39dd7-3275-40dd-a0cb-a7eda4a71bcc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:12:55.309590 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:55.309590 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:55.309590 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:55.309590 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:55.309590 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:55.309590 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:55.309590 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:55.309590 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:55.309590 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:55.309590 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:55.309590 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:55.309590 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:12:55.309590 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:12:55.309590 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:12:55.311238 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:12:55.311238 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:55.311238 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:55.311238 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:55.311238 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:55.311238 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:55.311238 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:55.311238 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:55.311238 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:55.311238 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:55.311238 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:55.311238 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:55.311238 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:55.311238 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:55.311238 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:55.311238 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:55.311238 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:55.312965 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:12:55.312965 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:12:55.312965 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:12:55.312965 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:12:55.312965 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:12:55.312965 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:12:55.312965 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:12:55.312965 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:12:55.315445 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32270a1-57cc-4340-b754-391b53b4b954 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:55.318841 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 295/608] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:12:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:55.320382 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f39dd7-3275-40dd-a0cb-a7eda4a71bcc None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:55.340729 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 314/609] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:12:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:55.342890 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32270a1-57cc-4340-b754-391b53b4b954 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:55.410591 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:55.478311 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f39dd7-3275-40dd-a0cb-a7eda4a71bcc None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:55.530422 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e993e327-0628-419a-9740-c89ecc59d277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:55.531817 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993e327-0628-419a-9740-c89ecc59d277 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:55.532270 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993e327-0628-419a-9740-c89ecc59d277 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:55.532687 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993e327-0628-419a-9740-c89ecc59d277 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:55.533654 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e993e327-0628-419a-9740-c89ecc59d277 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:55.534115 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e993e327-0628-419a-9740-c89ecc59d277 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:55.535848 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e993e327-0628-419a-9740-c89ecc59d277 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:55.540883 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32270a1-57cc-4340-b754-391b53b4b954 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:55.557166 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 296/610] 10.222.0.87 () {56 vars in 1148 bytes} [Tue Jun 21 18:12:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:55.558776 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e993e327-0628-419a-9740-c89ecc59d277 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:55.613003 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d41b8e-f7fe-4871-b625-a7b50118763e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:55.614523 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d41b8e-f7fe-4871-b625-a7b50118763e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:55.615204 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d41b8e-f7fe-4871-b625-a7b50118763e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:55.616865 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d41b8e-f7fe-4871-b625-a7b50118763e None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:55.617675 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d41b8e-f7fe-4871-b625-a7b50118763e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:55.618375 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d41b8e-f7fe-4871-b625-a7b50118763e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:55.621358 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d41b8e-f7fe-4871-b625-a7b50118763e None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:55.649930 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 315/611] 10.222.0.87 () {56 vars in 1148 bytes} [Tue Jun 21 18:12:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:55.649930 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d41b8e-f7fe-4871-b625-a7b50118763e None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:55.679922 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:55.722410 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e993e327-0628-419a-9740-c89ecc59d277 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:55.740873 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:55.772550 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a378d78e-02a9-4bd6-8fb1-e0404b620bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:55.773424 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378d78e-02a9-4bd6-8fb1-e0404b620bab None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:55.773733 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378d78e-02a9-4bd6-8fb1-e0404b620bab None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:55.774315 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378d78e-02a9-4bd6-8fb1-e0404b620bab None admin] PATH_INFO: `/v3/users/swift` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:55.786071 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a378d78e-02a9-4bd6-8fb1-e0404b620bab None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:55.788507 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a378d78e-02a9-4bd6-8fb1-e0404b620bab None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:55.813360 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-a378d78e-02a9-4bd6-8fb1-e0404b620bab None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 21 18:12:55.813360 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:55.813360 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:55.813360 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:55.813360 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:55.813360 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:55.813360 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:55.813360 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:55.813360 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:55.813360 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:55.813360 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:55.813360 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:55.813360 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:12:55.813360 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:12:55.813360 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:12:55.813360 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:12:55.814610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:55.814610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:55.814610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:12:55.814610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:12:55.814610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:12:55.814610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:12:55.814610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:55.814610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:55.814610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:55.814610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:55.814610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:55.814610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:55.814610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:55.814610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:55.814610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:55.814610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:55.814610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:55.814610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:55.815532 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:55.815532 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:55.815532 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:12:55.815532 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:12:55.815532 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:12:55.815532 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:12:55.815532 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:12:55.815532 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:12:55.815532 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 21 18:12:55.815532 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:12:55.816333 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 297/612] 10.222.0.87 () {56 vars in 1096 bytes} [Tue Jun 21 18:12:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:55.820029 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a378d78e-02a9-4bd6-8fb1-e0404b620bab None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:55.871739 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d41b8e-f7fe-4871-b625-a7b50118763e None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:55.930656 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242e2e2e-8f91-4031-9281-b554276d301b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:55.931589 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e2e2e-8f91-4031-9281-b554276d301b None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:55.931868 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e2e2e-8f91-4031-9281-b554276d301b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:55.932144 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e2e2e-8f91-4031-9281-b554276d301b None admin] PATH_INFO: `/v3/projects/5f30c3ddb5e44e9299ff228fe94c27fb/users/b683e6f78abd4986b415eb9ddf34cb8d/roles/b9dbf55e6f2947158c1d946f6ee9f907` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:55.952969 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242e2e2e-8f91-4031-9281-b554276d301b None admin] RBAC: Authorizing `identity:create_grant(project_id=5f30c3ddb5e44e9299ff228fe94c27fb, user_id=b683e6f78abd4986b415eb9ddf34cb8d, role_id=b9dbf55e6f2947158c1d946f6ee9f907)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:55.954498 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242e2e2e-8f91-4031-9281-b554276d301b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:55.974743 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 316/613] 10.222.0.87 () {58 vars in 1439 bytes} [Tue Jun 21 18:12:55 2022] PUT /identity/v3/projects/5f30c3ddb5e44e9299ff228fe94c27fb/users/b683e6f78abd4986b415eb9ddf34cb8d/roles/b9dbf55e6f2947158c1d946f6ee9f907 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:12:55.982931 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccaa3f3-c595-4fc3-970d-87aafb8d9300 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:55.983369 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccaa3f3-c595-4fc3-970d-87aafb8d9300 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:55.983765 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccaa3f3-c595-4fc3-970d-87aafb8d9300 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:55.993695 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 317/614] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:12:55 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:12:55.995287 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccaa3f3-c595-4fc3-970d-87aafb8d9300 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:56.038215 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a378d78e-02a9-4bd6-8fb1-e0404b620bab None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:56.100506 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae14084-5f78-4276-bccc-e9154a6bf617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:56.102661 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae14084-5f78-4276-bccc-e9154a6bf617 None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:56.103223 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae14084-5f78-4276-bccc-e9154a6bf617 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:56.103815 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae14084-5f78-4276-bccc-e9154a6bf617 None admin] PATH_INFO: `/v3/projects/5f30c3ddb5e44e9299ff228fe94c27fb/users/480e2529015c4c2fb4d6cd1985009fde/roles/b9dbf55e6f2947158c1d946f6ee9f907` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:56.130769 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:56.135047 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae14084-5f78-4276-bccc-e9154a6bf617 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f30c3ddb5e44e9299ff228fe94c27fb, user_id=480e2529015c4c2fb4d6cd1985009fde, role_id=b9dbf55e6f2947158c1d946f6ee9f907)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:56.137966 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae14084-5f78-4276-bccc-e9154a6bf617 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:56.161483 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 298/615] 10.222.0.87 () {58 vars in 1439 bytes} [Tue Jun 21 18:12:55 2022] PUT /identity/v3/projects/5f30c3ddb5e44e9299ff228fe94c27fb/users/480e2529015c4c2fb4d6cd1985009fde/roles/b9dbf55e6f2947158c1d946f6ee9f907 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:12:56.163420 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42851e0-7c46-47bb-bfa2-d410d25d649d None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:56.164039 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42851e0-7c46-47bb-bfa2-d410d25d649d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:56.164595 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42851e0-7c46-47bb-bfa2-d410d25d649d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:56.203617 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b42851e0-7c46-47bb-bfa2-d410d25d649d None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:12:56.206094 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b42851e0-7c46-47bb-bfa2-d410d25d649d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:12:56.235935 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccaa3f3-c595-4fc3-970d-87aafb8d9300 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:56.248949 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:56.301805 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ea003a-e0e8-43bd-92bc-0ef48a2990da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:56.302519 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea003a-e0e8-43bd-92bc-0ef48a2990da None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:56.302809 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea003a-e0e8-43bd-92bc-0ef48a2990da None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:56.303097 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea003a-e0e8-43bd-92bc-0ef48a2990da None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:56.303987 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ea003a-e0e8-43bd-92bc-0ef48a2990da None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:56.304306 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ea003a-e0e8-43bd-92bc-0ef48a2990da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:56.306323 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ea003a-e0e8-43bd-92bc-0ef48a2990da None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:56.332100 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 318/616] 10.222.0.87 () {56 vars in 1135 bytes} [Tue Jun 21 18:12:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:56.341291 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ea003a-e0e8-43bd-92bc-0ef48a2990da None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:56.395288 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 299/617] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:12:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:12:56.399220 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c74b4-c5c3-4a08-8acc-2b985f42ea3a None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:56.399715 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c74b4-c5c3-4a08-8acc-2b985f42ea3a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:56.400225 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c74b4-c5c3-4a08-8acc-2b985f42ea3a None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:56.404839 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 300/618] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:12:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:12:56.412244 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ac014-ce4c-4b3b-a507-932f2c925653 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:56.413060 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ac014-ce4c-4b3b-a507-932f2c925653 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:56.414164 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ac014-ce4c-4b3b-a507-932f2c925653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:56.439348 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-125ac014-ce4c-4b3b-a507-932f2c925653 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:12:56.441544 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-125ac014-ce4c-4b3b-a507-932f2c925653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:12:56.593846 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 301/619] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:12:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:12:56.595309 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ea003a-e0e8-43bd-92bc-0ef48a2990da None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:56.595680 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125ac014-ce4c-4b3b-a507-932f2c925653 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:56.642474 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:56.677764 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b0925-b7d0-486e-aaee-5c98e825457b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:56.679695 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b0925-b7d0-486e-aaee-5c98e825457b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:56.680568 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b0925-b7d0-486e-aaee-5c98e825457b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:56.681257 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b0925-b7d0-486e-aaee-5c98e825457b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:56.710540 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7b0925-b7d0-486e-aaee-5c98e825457b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:56.715722 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7b0925-b7d0-486e-aaee-5c98e825457b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:56.732658 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:56.755497 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125ac014-ce4c-4b3b-a507-932f2c925653 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:56.761919 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-6f7b0925-b7d0-486e-aaee-5c98e825457b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:56.761919 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:56.761919 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:56.761919 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:56.761919 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:56.761919 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:56.761919 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:56.761919 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:56.761919 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:56.761919 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:56.761919 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:56.761919 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:56.761919 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:12:56.761919 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:12:56.761919 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:12:56.763964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:12:56.763964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:56.763964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:56.763964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:56.763964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:56.763964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:56.763964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:56.763964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:56.763964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:56.763964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:56.763964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:56.763964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:56.763964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:56.763964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:56.763964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:56.763964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:56.763964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:56.766036 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:12:56.766036 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:12:56.766036 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:56.766036 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:12:56.770175 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 319/620] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:12:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 429 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:12:56.772217 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b0925-b7d0-486e-aaee-5c98e825457b None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:56.801722 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546c6301-4f07-4dd6-b4e1-df43eb5d4cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:56.802679 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c6301-4f07-4dd6-b4e1-df43eb5d4cef None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:56.802980 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c6301-4f07-4dd6-b4e1-df43eb5d4cef None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:56.803342 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c6301-4f07-4dd6-b4e1-df43eb5d4cef None admin] PATH_INFO: `/v3/roles/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:56.818456 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546c6301-4f07-4dd6-b4e1-df43eb5d4cef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:12:56.827513 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-546c6301-4f07-4dd6-b4e1-df43eb5d4cef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:12:56.827513 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:56.827513 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:56.827513 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:56.827513 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:56.827513 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:56.827513 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:56.827513 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:56.827513 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:56.827513 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:56.827513 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:56.827513 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:56.827513 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:12:56.827513 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:12:56.827513 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:12:56.827513 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:12:56.829232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:12:56.829232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:12:56.829232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:12:56.829232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:12:56.829232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:56.829232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:56.829232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:56.829232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:56.829232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:56.829232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:56.829232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:56.829232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:56.829232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:56.829232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:56.829232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:56.829232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:56.830222 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:56.830222 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:56.830222 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:56.830222 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:56.830222 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:12:56.830222 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:12:56.830222 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:12:56.830222 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:12:56.830222 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:12:56.830222 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:12:56.830222 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:12:56.830222 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:12:56.830222 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 302/621] 10.222.0.87 () {56 vars in 1102 bytes} [Tue Jun 21 18:12:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:56.831001 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546c6301-4f07-4dd6-b4e1-df43eb5d4cef None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:56.954146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b0925-b7d0-486e-aaee-5c98e825457b None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:56.964670 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546c6301-4f07-4dd6-b4e1-df43eb5d4cef None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:57.010925 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a692a20-c1bf-4523-aea4-2f4f40e50141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:57.011176 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d66fd-db81-448a-a601-99b7827d29f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:57.011952 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d66fd-db81-448a-a601-99b7827d29f4 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:57.012191 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d66fd-db81-448a-a601-99b7827d29f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:57.012491 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d66fd-db81-448a-a601-99b7827d29f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:57.013360 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4d66fd-db81-448a-a601-99b7827d29f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:57.013720 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4d66fd-db81-448a-a601-99b7827d29f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:57.014359 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a692a20-c1bf-4523-aea4-2f4f40e50141 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:57.014914 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a692a20-c1bf-4523-aea4-2f4f40e50141 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:57.015449 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a692a20-c1bf-4523-aea4-2f4f40e50141 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:57.015816 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4d66fd-db81-448a-a601-99b7827d29f4 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:57.033783 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a692a20-c1bf-4523-aea4-2f4f40e50141 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:12:57.034006 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 303/622] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:12:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:57.034796 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d66fd-db81-448a-a601-99b7827d29f4 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:57.039707 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:57.048787 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-4a692a20-c1bf-4523-aea4-2f4f40e50141 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:12:57.048787 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:57.048787 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:57.048787 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:57.048787 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:57.048787 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:57.048787 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:57.048787 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:57.048787 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:57.048787 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:57.048787 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:57.048787 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:57.048787 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:12:57.048787 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:12:57.048787 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:12:57.048787 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:12:57.050203 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:12:57.050203 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:12:57.050203 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:12:57.050203 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:12:57.050203 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:57.050203 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:57.050203 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:57.050203 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:57.050203 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:57.050203 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:57.050203 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:57.050203 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:57.050203 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:57.050203 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:57.050203 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:57.050203 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:57.051221 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:57.051221 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:57.051221 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:57.051221 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:57.051221 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:12:57.051221 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:12:57.051221 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:12:57.051221 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:12:57.051221 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:12:57.051221 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:12:57.051221 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:12:57.051221 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:12:57.055492 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 320/623] 10.222.0.87 () {56 vars in 1102 bytes} [Tue Jun 21 18:12:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:57.056932 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a692a20-c1bf-4523-aea4-2f4f40e50141 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:57.114357 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:57.190948 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d66fd-db81-448a-a601-99b7827d29f4 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:57.240424 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a692a20-c1bf-4523-aea4-2f4f40e50141 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:57.250791 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e86b7-3d4a-4808-aa74-8438bb822247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:57.252386 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e86b7-3d4a-4808-aa74-8438bb822247 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:57.253148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e86b7-3d4a-4808-aa74-8438bb822247 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:57.254054 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e86b7-3d4a-4808-aa74-8438bb822247 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:57.255856 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739e86b7-3d4a-4808-aa74-8438bb822247 None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:57.256706 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739e86b7-3d4a-4808-aa74-8438bb822247 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:57.259545 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739e86b7-3d4a-4808-aa74-8438bb822247 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:57.276933 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 304/624] 10.222.0.87 () {56 vars in 1103 bytes} [Tue Jun 21 18:12:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:57.278982 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e86b7-3d4a-4808-aa74-8438bb822247 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:57.306332 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3846a30a-26fc-4ea6-babd-c41d953897d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:57.307628 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3846a30a-26fc-4ea6-babd-c41d953897d8 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:57.308226 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3846a30a-26fc-4ea6-babd-c41d953897d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:57.308699 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3846a30a-26fc-4ea6-babd-c41d953897d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:57.322006 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3846a30a-26fc-4ea6-babd-c41d953897d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:57.324791 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3846a30a-26fc-4ea6-babd-c41d953897d8 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:57.352805 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-3846a30a-26fc-4ea6-babd-c41d953897d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:12:57.352805 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:57.352805 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:57.352805 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:57.352805 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:57.352805 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:57.352805 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:57.352805 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:57.352805 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:57.352805 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:57.352805 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:57.352805 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:57.352805 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:12:57.352805 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:12:57.352805 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:12:57.354700 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:12:57.354700 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:57.354700 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:57.354700 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:57.354700 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:57.354700 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:57.354700 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:57.354700 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:57.354700 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:57.354700 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:57.354700 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:57.354700 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:57.354700 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:57.354700 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:57.354700 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:57.354700 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:57.354700 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:57.358924 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:12:57.358924 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:12:57.358924 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:12:57.358924 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:12:57.358924 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:12:57.358924 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:12:57.358924 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:12:57.358924 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:12:57.358924 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 321/625] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:12:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:57.358924 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3846a30a-26fc-4ea6-babd-c41d953897d8 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:57.417054 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:57.486069 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e86b7-3d4a-4808-aa74-8438bb822247 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:57.547679 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854bbab9-d79e-44f0-ad21-81ff07214a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:57.548626 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854bbab9-d79e-44f0-ad21-81ff07214a70 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:57.549057 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854bbab9-d79e-44f0-ad21-81ff07214a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:57.549223 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:57.549550 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854bbab9-d79e-44f0-ad21-81ff07214a70 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:57.551342 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854bbab9-d79e-44f0-ad21-81ff07214a70 None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:57.551956 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854bbab9-d79e-44f0-ad21-81ff07214a70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:57.554523 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854bbab9-d79e-44f0-ad21-81ff07214a70 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:57.561755 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3846a30a-26fc-4ea6-babd-c41d953897d8 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:57.575477 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 305/626] 10.222.0.87 () {56 vars in 1103 bytes} [Tue Jun 21 18:12:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:57.575477 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854bbab9-d79e-44f0-ad21-81ff07214a70 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:57.634889 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4cce4-02fd-4b52-b236-f53717ea8097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:57.636218 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4cce4-02fd-4b52-b236-f53717ea8097 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:57.636693 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4cce4-02fd-4b52-b236-f53717ea8097 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:57.637165 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4cce4-02fd-4b52-b236-f53717ea8097 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:57.658115 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a4cce4-02fd-4b52-b236-f53717ea8097 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:57.660667 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a4cce4-02fd-4b52-b236-f53717ea8097 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:57.695986 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-49a4cce4-02fd-4b52-b236-f53717ea8097 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:57.695986 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:57.695986 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:57.695986 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:57.695986 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:57.695986 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:57.695986 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:57.695986 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:57.695986 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:57.695986 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:57.695986 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:57.695986 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:57.695986 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:12:57.695986 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:12:57.695986 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:12:57.697745 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:12:57.697745 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:57.697745 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:57.697745 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:57.697745 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:57.697745 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:57.697745 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:57.697745 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:57.697745 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:57.697745 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:57.697745 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:57.697745 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:57.697745 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:57.697745 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:57.697745 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:57.697745 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:57.697745 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:57.700713 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:12:57.700713 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:12:57.700713 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:57.700713 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:12:57.700713 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 322/627] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:12:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:57.701259 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4cce4-02fd-4b52-b236-f53717ea8097 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:57.783553 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854bbab9-d79e-44f0-ad21-81ff07214a70 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:57.846573 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:57.864872 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f80b4b-a432-4acc-9ed5-02c2b6e41040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:57.866601 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f80b4b-a432-4acc-9ed5-02c2b6e41040 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:57.867363 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f80b4b-a432-4acc-9ed5-02c2b6e41040 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:57.868498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f80b4b-a432-4acc-9ed5-02c2b6e41040 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:57.870530 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f80b4b-a432-4acc-9ed5-02c2b6e41040 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:57.871400 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f80b4b-a432-4acc-9ed5-02c2b6e41040 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:57.874688 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f80b4b-a432-4acc-9ed5-02c2b6e41040 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:57.883935 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4cce4-02fd-4b52-b236-f53717ea8097 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:57.906209 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 306/628] 10.222.0.87 () {56 vars in 1148 bytes} [Tue Jun 21 18:12:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:57.908474 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f80b4b-a432-4acc-9ed5-02c2b6e41040 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:57.939585 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f513fd77-280d-4756-95e3-ffb8d207c70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:57.940863 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f513fd77-280d-4756-95e3-ffb8d207c70f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:57.941212 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f513fd77-280d-4756-95e3-ffb8d207c70f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:57.941691 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f513fd77-280d-4756-95e3-ffb8d207c70f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:57.958085 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f513fd77-280d-4756-95e3-ffb8d207c70f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:57.960722 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f513fd77-280d-4756-95e3-ffb8d207c70f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:57.966838 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:58.001120 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-f513fd77-280d-4756-95e3-ffb8d207c70f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:58.001120 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:58.001120 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:58.001120 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:58.001120 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:58.001120 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:58.001120 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:58.001120 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:58.001120 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:58.001120 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:58.001120 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:58.001120 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:58.001120 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:12:58.001120 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:12:58.001120 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:12:58.003143 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:12:58.003143 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:58.003143 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:58.003143 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:58.003143 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:58.003143 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:58.003143 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:58.003143 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:58.003143 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:58.003143 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:58.003143 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:58.003143 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:58.003143 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:58.003143 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:58.003143 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:58.003143 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:58.003143 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:58.005119 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:12:58.005119 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:12:58.005119 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:58.005119 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:12:58.005119 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 323/629] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:12:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:58.005670 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f513fd77-280d-4756-95e3-ffb8d207c70f None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:58.118616 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f80b4b-a432-4acc-9ed5-02c2b6e41040 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:58.188174 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16668f5-7de8-4018-83f0-79b612c03c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:58.189747 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16668f5-7de8-4018-83f0-79b612c03c1a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:58.190488 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16668f5-7de8-4018-83f0-79b612c03c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:58.191201 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16668f5-7de8-4018-83f0-79b612c03c1a None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:58.192720 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16668f5-7de8-4018-83f0-79b612c03c1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:58.193444 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16668f5-7de8-4018-83f0-79b612c03c1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:58.196472 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16668f5-7de8-4018-83f0-79b612c03c1a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:58.219791 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 307/630] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:12:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:58.221695 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16668f5-7de8-4018-83f0-79b612c03c1a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:58.226264 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f513fd77-280d-4756-95e3-ffb8d207c70f None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:58.295787 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8714173f-747c-4f05-affe-2322c89dfd87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:58.296983 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714173f-747c-4f05-affe-2322c89dfd87 None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:58.297376 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714173f-747c-4f05-affe-2322c89dfd87 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:58.297808 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714173f-747c-4f05-affe-2322c89dfd87 None admin] PATH_INFO: `/v3/projects/5f30c3ddb5e44e9299ff228fe94c27fb/users/515b3571045e4fe29565307c8ecc10b2/roles/b9dbf55e6f2947158c1d946f6ee9f907` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:58.318954 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8714173f-747c-4f05-affe-2322c89dfd87 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f30c3ddb5e44e9299ff228fe94c27fb, user_id=515b3571045e4fe29565307c8ecc10b2, role_id=b9dbf55e6f2947158c1d946f6ee9f907)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:58.320502 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8714173f-747c-4f05-affe-2322c89dfd87 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:58.339966 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 324/631] 10.222.0.87 () {58 vars in 1439 bytes} [Tue Jun 21 18:12:58 2022] PUT /identity/v3/projects/5f30c3ddb5e44e9299ff228fe94c27fb/users/515b3571045e4fe29565307c8ecc10b2/roles/b9dbf55e6f2947158c1d946f6ee9f907 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:12:58.341242 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8714173f-747c-4f05-affe-2322c89dfd87 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:58.347612 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:58.410196 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:58.455039 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16668f5-7de8-4018-83f0-79b612c03c1a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:58.497847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8714173f-747c-4f05-affe-2322c89dfd87 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:58.526296 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea8d014-6a12-4619-bc32-a8f5c1be3e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:58.528082 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea8d014-6a12-4619-bc32-a8f5c1be3e52 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:58.528547 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea8d014-6a12-4619-bc32-a8f5c1be3e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:58.529040 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea8d014-6a12-4619-bc32-a8f5c1be3e52 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:58.530511 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea8d014-6a12-4619-bc32-a8f5c1be3e52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:58.531043 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea8d014-6a12-4619-bc32-a8f5c1be3e52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:58.533784 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea8d014-6a12-4619-bc32-a8f5c1be3e52 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:58.554962 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e75948-4535-43cd-beb7-29adccc8b7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:58.556207 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e75948-4535-43cd-beb7-29adccc8b7e7 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:58.556772 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e75948-4535-43cd-beb7-29adccc8b7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:58.557269 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e75948-4535-43cd-beb7-29adccc8b7e7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:58.558685 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 308/632] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:12:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:58.568001 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e75948-4535-43cd-beb7-29adccc8b7e7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:58.568302 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea8d014-6a12-4619-bc32-a8f5c1be3e52 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:58.570875 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e75948-4535-43cd-beb7-29adccc8b7e7 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:58.594685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-14e75948-4535-43cd-beb7-29adccc8b7e7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:12:58.594685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:58.594685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:58.594685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:58.594685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:58.594685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:58.594685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:58.594685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:58.594685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:58.594685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:58.594685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:58.594685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:58.594685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:12:58.594685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:12:58.594685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:12:58.594685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:12:58.595764 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:58.595764 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:58.595764 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:12:58.595764 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:12:58.595764 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:12:58.595764 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:12:58.595764 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:58.595764 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:58.595764 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:58.595764 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:58.595764 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:58.595764 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:58.595764 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:58.595764 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:58.595764 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:58.595764 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:58.595764 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:58.595764 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:58.596894 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:58.596894 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:58.596894 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:12:58.596894 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:12:58.596894 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:12:58.596894 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:12:58.596894 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:12:58.596894 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:12:58.596894 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:12:58.596894 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:12:58.599371 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 325/633] 10.222.0.87 () {56 vars in 1093 bytes} [Tue Jun 21 18:12:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:58.608474 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e75948-4535-43cd-beb7-29adccc8b7e7 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:58.852403 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea8d014-6a12-4619-bc32-a8f5c1be3e52 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:58.865770 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:58.929351 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a695d2de-9da5-42f6-aa17-c836d5185c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:58.930712 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695d2de-9da5-42f6-aa17-c836d5185c3f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:58.931314 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695d2de-9da5-42f6-aa17-c836d5185c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:58.932041 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695d2de-9da5-42f6-aa17-c836d5185c3f None admin] PATH_INFO: `/v3/users/glance` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:58.934111 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e75948-4535-43cd-beb7-29adccc8b7e7 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:58.948066 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a695d2de-9da5-42f6-aa17-c836d5185c3f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:58.950205 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:58.959011 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a695d2de-9da5-42f6-aa17-c836d5185c3f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:58.989012 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-a695d2de-9da5-42f6-aa17-c836d5185c3f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 21 18:12:58.989012 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:58.989012 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:58.989012 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:58.989012 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:58.989012 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:58.989012 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:58.989012 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:58.989012 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:58.989012 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:58.989012 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:58.989012 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:58.989012 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:12:58.989012 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:12:58.989012 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:12:58.989012 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:12:58.991123 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:58.991123 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:58.991123 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:12:58.991123 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:12:58.991123 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:12:58.991123 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:12:58.991123 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:58.991123 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:58.991123 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:58.991123 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:58.991123 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:58.991123 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:58.991123 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:58.991123 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:58.991123 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:58.991123 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:58.991123 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:58.991123 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:58.993275 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:58.993275 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:58.993275 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:12:58.993275 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:12:58.993275 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:12:58.993275 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:12:58.993275 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:12:58.993275 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:12:58.993275 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 21 18:12:58.993275 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:12:58.998510 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 309/634] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:12:58 2022] GET /identity/v3/users/glance => generated 84 bytes in 427 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:59.015627 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a695d2de-9da5-42f6-aa17-c836d5185c3f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:59.024948 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e3db8-39aa-47f0-9ddf-9ac9dc5fe0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:59.026799 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e3db8-39aa-47f0-9ddf-9ac9dc5fe0cd None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:59.027690 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e3db8-39aa-47f0-9ddf-9ac9dc5fe0cd None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:59.028446 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e3db8-39aa-47f0-9ddf-9ac9dc5fe0cd None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:59.030445 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48e3db8-39aa-47f0-9ddf-9ac9dc5fe0cd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:59.031354 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48e3db8-39aa-47f0-9ddf-9ac9dc5fe0cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:59.034513 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48e3db8-39aa-47f0-9ddf-9ac9dc5fe0cd None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:59.075547 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 326/635] 10.222.0.87 () {56 vars in 1133 bytes} [Tue Jun 21 18:12:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 467 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:12:59.095731 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e3db8-39aa-47f0-9ddf-9ac9dc5fe0cd None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:59.246625 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a695d2de-9da5-42f6-aa17-c836d5185c3f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:59.277012 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e3db8-39aa-47f0-9ddf-9ac9dc5fe0cd None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:59.316649 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ef506-2011-48b7-bcfb-7a132dc0b9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:59.318634 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ef506-2011-48b7-bcfb-7a132dc0b9b5 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:59.318814 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4349f4a0-a04f-44ac-a2d4-e322495b3109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:59.319508 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349f4a0-a04f-44ac-a2d4-e322495b3109 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:59.319836 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349f4a0-a04f-44ac-a2d4-e322495b3109 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:59.320184 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349f4a0-a04f-44ac-a2d4-e322495b3109 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:59.321039 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ef506-2011-48b7-bcfb-7a132dc0b9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:59.321997 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ef506-2011-48b7-bcfb-7a132dc0b9b5 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:59.324321 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588ef506-2011-48b7-bcfb-7a132dc0b9b5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:12:59.325324 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588ef506-2011-48b7-bcfb-7a132dc0b9b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:59.328328 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588ef506-2011-48b7-bcfb-7a132dc0b9b5 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:59.361176 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:59.373034 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 310/636] 10.222.0.87 () {56 vars in 1137 bytes} [Tue Jun 21 18:12:59 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:12:59.376156 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d50ce-1423-4e46-8392-feea4cd03427 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:59.377091 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d50ce-1423-4e46-8392-feea4cd03427 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:59.377934 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d50ce-1423-4e46-8392-feea4cd03427 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:59.383555 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 311/637] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:12:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:12:59.385732 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890d50ce-1423-4e46-8392-feea4cd03427 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:59.409222 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4349f4a0-a04f-44ac-a2d4-e322495b3109 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:59.410858 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4349f4a0-a04f-44ac-a2d4-e322495b3109 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:59.448971 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:59.535355 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 327/638] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:12:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:12:59.536551 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642bb92-a25c-41cf-b4fa-2209e1944293 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:59.536853 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642bb92-a25c-41cf-b4fa-2209e1944293 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:59.537141 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642bb92-a25c-41cf-b4fa-2209e1944293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:59.561732 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-0642bb92-a25c-41cf-b4fa-2209e1944293 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:12:59.563008 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-0642bb92-a25c-41cf-b4fa-2209e1944293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:12:59.592023 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890d50ce-1423-4e46-8392-feea4cd03427 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:59.647192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db50770b-9ea0-4098-a1d0-fe99157dba49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:59.648967 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50770b-9ea0-4098-a1d0-fe99157dba49 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:59.649865 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50770b-9ea0-4098-a1d0-fe99157dba49 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:59.650050 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:59.650892 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50770b-9ea0-4098-a1d0-fe99157dba49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:59.669672 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 328/639] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:12:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:12:59.669672 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0642bb92-a25c-41cf-b4fa-2209e1944293 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:59.786156 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db50770b-9ea0-4098-a1d0-fe99157dba49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:59.788425 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db50770b-9ea0-4098-a1d0-fe99157dba49 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:59.820887 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0642bb92-a25c-41cf-b4fa-2209e1944293 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:12:59.832431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:12:59.897217 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a183204-5ad7-4067-8296-8de9801d923d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:12:59.898584 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a183204-5ad7-4067-8296-8de9801d923d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:12:59.899065 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a183204-5ad7-4067-8296-8de9801d923d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:12:59.899549 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a183204-5ad7-4067-8296-8de9801d923d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:12:59.921036 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a183204-5ad7-4067-8296-8de9801d923d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:12:59.923870 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a183204-5ad7-4067-8296-8de9801d923d None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:12:59.952279 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 312/640] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:12:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:12:59.954141 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db50770b-9ea0-4098-a1d0-fe99157dba49 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:12:59.959795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-0a183204-5ad7-4067-8296-8de9801d923d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:59.959795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:12:59.959795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:12:59.959795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:12:59.959795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:12:59.959795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:12:59.959795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:12:59.959795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:12:59.959795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:12:59.959795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:12:59.959795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:12:59.959795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:12:59.959795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:12:59.959795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:12:59.959795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:12:59.960973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:12:59.960973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:12:59.960973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:12:59.960973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:12:59.960973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:12:59.960973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:12:59.960973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:12:59.960973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:12:59.960973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:12:59.960973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:12:59.960973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:12:59.960973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:12:59.960973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:12:59.960973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:12:59.960973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:12:59.960973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:12:59.960973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:12:59.962361 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:12:59.962361 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:12:59.962361 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:12:59.962361 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:12:59.970201 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 329/641] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:12:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:12:59.971412 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a183204-5ad7-4067-8296-8de9801d923d None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:00.058513 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:00.137528 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db50770b-9ea0-4098-a1d0-fe99157dba49 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:00.155301 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a183204-5ad7-4067-8296-8de9801d923d None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:00.187878 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:00.201754 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d6e016-d075-41af-b90d-9fcc3ce2cfa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:00.203109 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6e016-d075-41af-b90d-9fcc3ce2cfa6 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:00.203615 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6e016-d075-41af-b90d-9fcc3ce2cfa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:00.204099 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6e016-d075-41af-b90d-9fcc3ce2cfa6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:00.222682 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d6e016-d075-41af-b90d-9fcc3ce2cfa6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:13:00.227888 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1bf9d5-5b55-45c5-97e7-da2839ce765c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:00.228733 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1bf9d5-5b55-45c5-97e7-da2839ce765c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:00.229130 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1bf9d5-5b55-45c5-97e7-da2839ce765c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:00.229477 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1bf9d5-5b55-45c5-97e7-da2839ce765c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:00.239295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-89d6e016-d075-41af-b90d-9fcc3ce2cfa6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:13:00.239295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:00.239295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:00.239295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:00.239295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:00.239295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:00.239295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:00.239295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:00.239295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:00.239295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:00.239295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:00.239295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:00.239295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:13:00.239295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:13:00.239295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:13:00.239295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:13:00.240431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:13:00.240431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:13:00.240431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:13:00.240431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:13:00.240431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:00.240431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:00.240431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:00.240431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:00.240431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:00.240431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:00.240431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:00.240431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:00.240431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:00.240431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:00.240431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:00.240431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:00.241308 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:00.241308 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:00.241308 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:00.241308 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:00.241308 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:13:00.241308 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:13:00.241308 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:13:00.241308 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:13:00.241308 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:13:00.241308 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:13:00.241308 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:13:00.241308 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:00.244436 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 313/642] 10.222.0.87 () {56 vars in 1102 bytes} [Tue Jun 21 18:12:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:00.244436 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d6e016-d075-41af-b90d-9fcc3ce2cfa6 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:00.250928 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1bf9d5-5b55-45c5-97e7-da2839ce765c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:00.252490 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1bf9d5-5b55-45c5-97e7-da2839ce765c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:00.274977 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-fe1bf9d5-5b55-45c5-97e7-da2839ce765c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:00.274977 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:00.274977 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:00.274977 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:00.274977 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:00.274977 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:00.274977 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:00.274977 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:00.274977 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:00.274977 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:00.274977 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:00.274977 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:00.274977 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:13:00.274977 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:13:00.274977 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:13:00.275973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:13:00.275973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:00.275973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:00.275973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:00.275973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:00.275973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:00.275973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:00.275973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:00.275973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:00.275973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:00.275973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:00.275973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:00.275973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:00.275973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:00.275973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:00.275973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:00.275973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:00.277225 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:13:00.277225 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:13:00.277225 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:00.277225 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:00.277225 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 330/643] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:12:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:00.279345 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6714db4-ee28-49d0-b791-b318af6b568e None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:00.279345 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6714db4-ee28-49d0-b791-b318af6b568e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:00.279345 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6714db4-ee28-49d0-b791-b318af6b568e None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:00.280569 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 331/644] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:13:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:13:00.281353 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6714db4-ee28-49d0-b791-b318af6b568e None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:00.427720 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:00.437687 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d6e016-d075-41af-b90d-9fcc3ce2cfa6 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:00.437936 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6714db4-ee28-49d0-b791-b318af6b568e None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:00.506571 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03afbc7-914a-4726-8bbb-f8add2a5c9ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:00.507287 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03afbc7-914a-4726-8bbb-f8add2a5c9ab None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:00.507287 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03afbc7-914a-4726-8bbb-f8add2a5c9ab None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:00.509372 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03afbc7-914a-4726-8bbb-f8add2a5c9ab None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:00.509517 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03afbc7-914a-4726-8bbb-f8add2a5c9ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:00.510017 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03afbc7-914a-4726-8bbb-f8add2a5c9ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:00.512925 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03afbc7-914a-4726-8bbb-f8add2a5c9ab None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:00.517189 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aeacef-8e7e-4d4f-9da5-bc510d7e9aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:00.518296 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aeacef-8e7e-4d4f-9da5-bc510d7e9aba None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:00.518734 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aeacef-8e7e-4d4f-9da5-bc510d7e9aba None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:00.519105 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aeacef-8e7e-4d4f-9da5-bc510d7e9aba None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:00.520427 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aeacef-8e7e-4d4f-9da5-bc510d7e9aba None admin] RBAC: Adding query filter params (name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:00.521265 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aeacef-8e7e-4d4f-9da5-bc510d7e9aba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:00.523920 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aeacef-8e7e-4d4f-9da5-bc510d7e9aba None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:00.546098 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 314/645] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:13:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:00.546502 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 332/646] 10.222.0.87 () {56 vars in 1103 bytes} [Tue Jun 21 18:13:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:00.546502 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aeacef-8e7e-4d4f-9da5-bc510d7e9aba None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:00.547100 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4158df-f761-4282-a6ad-9cf5b66d715a None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:00.547100 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4158df-f761-4282-a6ad-9cf5b66d715a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:00.547100 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4158df-f761-4282-a6ad-9cf5b66d715a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:00.566938 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-0b4158df-f761-4282-a6ad-9cf5b66d715a None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:13:00.568087 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0b4158df-f761-4282-a6ad-9cf5b66d715a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:13:00.643153 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:00.687309 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 315/647] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:13:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:13:00.688124 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4158df-f761-4282-a6ad-9cf5b66d715a None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:00.737800 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aeacef-8e7e-4d4f-9da5-bc510d7e9aba None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:00.779537 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c990e585-b6b6-497d-8dfe-5751eb646958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:00.780403 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c990e585-b6b6-497d-8dfe-5751eb646958 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:00.780688 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c990e585-b6b6-497d-8dfe-5751eb646958 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:00.780968 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c990e585-b6b6-497d-8dfe-5751eb646958 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:00.781777 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c990e585-b6b6-497d-8dfe-5751eb646958 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:00.782336 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c990e585-b6b6-497d-8dfe-5751eb646958 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:00.783889 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c990e585-b6b6-497d-8dfe-5751eb646958 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:00.798828 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 333/648] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:13:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:00.800288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c990e585-b6b6-497d-8dfe-5751eb646958 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:00.825397 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4158df-f761-4282-a6ad-9cf5b66d715a None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:00.841871 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:00.878248 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73376bad-df90-4632-9882-cd2e072121f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:00.879685 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73376bad-df90-4632-9882-cd2e072121f0 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:00.880185 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73376bad-df90-4632-9882-cd2e072121f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:00.880677 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73376bad-df90-4632-9882-cd2e072121f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:00.891774 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73376bad-df90-4632-9882-cd2e072121f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:00.893306 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73376bad-df90-4632-9882-cd2e072121f0 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:00.908577 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-73376bad-df90-4632-9882-cd2e072121f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:13:00.908577 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:00.908577 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:00.908577 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:00.908577 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:00.908577 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:00.908577 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:00.908577 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:00.908577 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:00.908577 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:00.908577 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:00.908577 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:00.908577 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:13:00.908577 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:13:00.908577 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:13:00.909554 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:13:00.909554 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:00.909554 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:00.909554 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:00.909554 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:00.909554 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:00.909554 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:00.909554 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:00.909554 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:00.909554 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:00.909554 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:00.909554 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:00.909554 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:00.909554 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:00.909554 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:00.909554 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:00.909554 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:00.916360 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:13:00.916360 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:13:00.916360 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:13:00.916360 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:13:00.916360 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:13:00.916360 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:13:00.916360 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:13:00.916360 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:00.916360 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 316/649] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:13:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:00.916360 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73376bad-df90-4632-9882-cd2e072121f0 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:00.953719 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c990e585-b6b6-497d-8dfe-5751eb646958 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:00.974408 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:01.004687 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bacc4b-5450-4540-80f9-ad167db8187f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:01.005519 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bacc4b-5450-4540-80f9-ad167db8187f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:01.005865 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bacc4b-5450-4540-80f9-ad167db8187f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:01.006174 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bacc4b-5450-4540-80f9-ad167db8187f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:01.020093 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bacc4b-5450-4540-80f9-ad167db8187f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:01.022201 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bacc4b-5450-4540-80f9-ad167db8187f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:01.043308 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-e8bacc4b-5450-4540-80f9-ad167db8187f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:01.043308 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:01.043308 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:01.043308 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:01.043308 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:01.043308 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:01.043308 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:01.043308 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:01.043308 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:01.043308 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:01.043308 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:01.043308 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:01.043308 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:13:01.043308 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:13:01.043308 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:13:01.044305 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:13:01.044305 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:01.044305 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:01.044305 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:01.044305 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:01.044305 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:01.044305 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:01.044305 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:01.044305 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:01.044305 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:01.044305 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:01.044305 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:01.044305 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:01.044305 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:01.044305 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:01.044305 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:01.044305 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:01.050505 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:13:01.050505 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:13:01.050505 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:01.050505 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:01.050505 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 334/650] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:13:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:01.050505 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bacc4b-5450-4540-80f9-ad167db8187f None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:01.061414 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73376bad-df90-4632-9882-cd2e072121f0 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:01.107736 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cfc45e-7705-4ada-9900-a5298d620999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:01.108585 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cfc45e-7705-4ada-9900-a5298d620999 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:01.108876 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cfc45e-7705-4ada-9900-a5298d620999 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:01.109262 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cfc45e-7705-4ada-9900-a5298d620999 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:01.129726 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cfc45e-7705-4ada-9900-a5298d620999 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:13:01.138152 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-a9cfc45e-7705-4ada-9900-a5298d620999 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:13:01.138152 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:01.138152 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:01.138152 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:01.138152 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:01.138152 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:01.138152 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:01.138152 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:01.138152 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:01.138152 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:01.138152 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:01.138152 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:01.138152 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:13:01.138152 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:13:01.138152 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:13:01.138152 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:13:01.139192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:13:01.139192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:13:01.139192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:13:01.139192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:13:01.139192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:01.139192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:01.139192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:01.139192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:01.139192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:01.139192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:01.139192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:01.139192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:01.139192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:01.139192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:01.139192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:01.139192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:01.140283 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:01.140283 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:01.140283 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:01.140283 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:01.140283 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:13:01.140283 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:13:01.140283 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:13:01.140283 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:13:01.140283 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:13:01.140283 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:13:01.140283 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:13:01.140283 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:01.140283 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 317/651] 10.222.0.87 () {56 vars in 1102 bytes} [Tue Jun 21 18:13:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:01.140981 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cfc45e-7705-4ada-9900-a5298d620999 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:01.147695 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:01.191824 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bacc4b-5450-4540-80f9-ad167db8187f None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:01.234596 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23edd647-a1d6-450e-b352-7102de0ca4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:01.235356 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23edd647-a1d6-450e-b352-7102de0ca4bd None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:01.235621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23edd647-a1d6-450e-b352-7102de0ca4bd None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:01.235998 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23edd647-a1d6-450e-b352-7102de0ca4bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:01.245384 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23edd647-a1d6-450e-b352-7102de0ca4bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:01.247134 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23edd647-a1d6-450e-b352-7102de0ca4bd None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:01.263638 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:01.269690 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-23edd647-a1d6-450e-b352-7102de0ca4bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:13:01.269690 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:01.269690 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:01.269690 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:01.269690 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:01.269690 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:01.269690 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:01.269690 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:01.269690 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:01.269690 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:01.269690 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:01.269690 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:01.269690 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:13:01.269690 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:13:01.269690 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:13:01.271592 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:13:01.271592 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:01.271592 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:01.271592 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:01.271592 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:01.271592 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:01.271592 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:01.271592 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:01.271592 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:01.271592 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:01.271592 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:01.271592 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:01.271592 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:01.271592 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:01.271592 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:01.271592 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:01.271592 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:01.281413 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:13:01.281413 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:13:01.281413 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:13:01.281413 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:13:01.281413 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:13:01.281413 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:13:01.281413 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:13:01.281413 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:01.281413 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 335/652] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:13:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:01.281413 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23edd647-a1d6-450e-b352-7102de0ca4bd None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:01.287405 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cfc45e-7705-4ada-9900-a5298d620999 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:01.343737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4a68f4-f437-4b54-b6eb-bb267cfecf73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:01.344736 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a68f4-f437-4b54-b6eb-bb267cfecf73 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:01.345156 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a68f4-f437-4b54-b6eb-bb267cfecf73 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:01.345566 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a68f4-f437-4b54-b6eb-bb267cfecf73 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:01.347099 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4a68f4-f437-4b54-b6eb-bb267cfecf73 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:01.347624 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4a68f4-f437-4b54-b6eb-bb267cfecf73 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:01.349153 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4a68f4-f437-4b54-b6eb-bb267cfecf73 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:01.374581 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 318/653] 10.222.0.87 () {56 vars in 1148 bytes} [Tue Jun 21 18:13:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:01.374581 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4a68f4-f437-4b54-b6eb-bb267cfecf73 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:01.477486 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23edd647-a1d6-450e-b352-7102de0ca4bd None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:01.539919 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:01.550909 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68c89ff-a27e-4e36-b456-376f0b5a566c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:01.551132 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4a68f4-f437-4b54-b6eb-bb267cfecf73 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:01.551504 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c89ff-a27e-4e36-b456-376f0b5a566c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:01.552013 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c89ff-a27e-4e36-b456-376f0b5a566c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:01.552841 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c89ff-a27e-4e36-b456-376f0b5a566c None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:01.554171 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68c89ff-a27e-4e36-b456-376f0b5a566c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:01.554705 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68c89ff-a27e-4e36-b456-376f0b5a566c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:01.557839 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68c89ff-a27e-4e36-b456-376f0b5a566c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:01.586386 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 336/654] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:13:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:01.588316 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68c89ff-a27e-4e36-b456-376f0b5a566c None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:01.626299 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d97f6ff-626a-4b5b-bac0-a685ef060357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:01.627490 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97f6ff-626a-4b5b-bac0-a685ef060357 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:01.627990 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97f6ff-626a-4b5b-bac0-a685ef060357 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:01.628464 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97f6ff-626a-4b5b-bac0-a685ef060357 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:01.629912 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d97f6ff-626a-4b5b-bac0-a685ef060357 None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:01.630411 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d97f6ff-626a-4b5b-bac0-a685ef060357 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:01.632849 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d97f6ff-626a-4b5b-bac0-a685ef060357 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:01.648554 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 319/655] 10.222.0.87 () {56 vars in 1103 bytes} [Tue Jun 21 18:13:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:01.649847 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d97f6ff-626a-4b5b-bac0-a685ef060357 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:01.683544 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:01.795393 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68c89ff-a27e-4e36-b456-376f0b5a566c None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:01.815411 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d97f6ff-626a-4b5b-bac0-a685ef060357 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:01.851861 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25118663-f29b-48e5-ae4b-28f8e31ed8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:01.852561 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25118663-f29b-48e5-ae4b-28f8e31ed8c8 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:01.852882 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25118663-f29b-48e5-ae4b-28f8e31ed8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:01.853158 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25118663-f29b-48e5-ae4b-28f8e31ed8c8 None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:01.854489 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25118663-f29b-48e5-ae4b-28f8e31ed8c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:01.854489 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25118663-f29b-48e5-ae4b-28f8e31ed8c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:01.856853 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25118663-f29b-48e5-ae4b-28f8e31ed8c8 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:01.868949 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a60a5c3-11ec-4e75-bccd-3269d34106c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:01.869778 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a60a5c3-11ec-4e75-bccd-3269d34106c6 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:01.870212 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a60a5c3-11ec-4e75-bccd-3269d34106c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:01.870869 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a60a5c3-11ec-4e75-bccd-3269d34106c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:01.871050 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 337/656] 10.222.0.87 () {56 vars in 1148 bytes} [Tue Jun 21 18:13:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:01.872239 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a60a5c3-11ec-4e75-bccd-3269d34106c6 None admin] RBAC: Adding query filter params (role.id=b9dbf55e6f2947158c1d946f6ee9f907, scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb, user.id=a1b3fa67905c4027b3656ac36e2b76c9) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:01.872571 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25118663-f29b-48e5-ae4b-28f8e31ed8c8 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:01.872798 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a60a5c3-11ec-4e75-bccd-3269d34106c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:01.875544 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a60a5c3-11ec-4e75-bccd-3269d34106c6 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:01.884045 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 320/657] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:13:01 2022] GET /identity/v3/role_assignments?user.id=a1b3fa67905c4027b3656ac36e2b76c9&scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb&role.id=b9dbf55e6f2947158c1d946f6ee9f907 => generated 608 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:01.886115 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a534b48-7ca7-47e8-91d0-42697dee78bf None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:01.886438 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a534b48-7ca7-47e8-91d0-42697dee78bf None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:01.886753 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a534b48-7ca7-47e8-91d0-42697dee78bf None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:01.888217 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 321/658] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:13:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:13:01.889268 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a534b48-7ca7-47e8-91d0-42697dee78bf None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:01.953098 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:02.020959 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25118663-f29b-48e5-ae4b-28f8e31ed8c8 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:02.029283 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:02.060736 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a534b48-7ca7-47e8-91d0-42697dee78bf None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:02.081510 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4408ef1-bf65-4616-8bf5-ee44a86be3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:02.082933 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4408ef1-bf65-4616-8bf5-ee44a86be3f9 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:02.083429 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4408ef1-bf65-4616-8bf5-ee44a86be3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:02.083928 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4408ef1-bf65-4616-8bf5-ee44a86be3f9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:02.100671 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4408ef1-bf65-4616-8bf5-ee44a86be3f9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:02.103678 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e00b7-8ea7-4f06-8a85-43f8340c03e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:02.103910 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4408ef1-bf65-4616-8bf5-ee44a86be3f9 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:02.104649 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e00b7-8ea7-4f06-8a85-43f8340c03e2 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:02.105005 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e00b7-8ea7-4f06-8a85-43f8340c03e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:02.105162 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e00b7-8ea7-4f06-8a85-43f8340c03e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:02.118801 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2e00b7-8ea7-4f06-8a85-43f8340c03e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:02.120380 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2e00b7-8ea7-4f06-8a85-43f8340c03e2 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:02.132911 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-f4408ef1-bf65-4616-8bf5-ee44a86be3f9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 21 18:13:02.132911 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:02.132911 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:02.132911 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:02.132911 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:02.132911 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:02.132911 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:02.132911 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:02.132911 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:02.132911 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:02.132911 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:02.132911 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:02.132911 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:13:02.132911 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:13:02.132911 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:13:02.132911 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:13:02.134224 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:02.134224 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:02.134224 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:13:02.134224 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:13:02.134224 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:13:02.134224 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:13:02.134224 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:02.134224 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:02.134224 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:02.134224 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:02.134224 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:02.134224 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:02.134224 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:02.134224 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:02.134224 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:02.134224 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:02.134224 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:02.134224 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:02.135168 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:02.135168 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:02.135168 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:13:02.135168 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:13:02.135168 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:13:02.135168 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:13:02.135168 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:13:02.135168 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:13:02.135168 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 21 18:13:02.135168 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:02.135741 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 338/659] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:13:01 2022] GET /identity/v3/users/cinder => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:02.136765 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4408ef1-bf65-4616-8bf5-ee44a86be3f9 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:02.141198 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-ff2e00b7-8ea7-4f06-8a85-43f8340c03e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:02.141198 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:02.141198 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:02.141198 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:02.141198 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:02.141198 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:02.141198 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:02.141198 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:02.141198 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:02.141198 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:02.141198 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:02.141198 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:02.141198 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:13:02.141198 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:13:02.141198 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:13:02.142395 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:13:02.142395 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:02.142395 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:02.142395 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:02.142395 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:02.142395 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:02.142395 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:02.142395 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:02.142395 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:02.142395 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:02.142395 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:02.142395 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:02.142395 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:02.142395 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:02.142395 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:02.142395 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:02.142395 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:02.144636 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:13:02.144636 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:13:02.144636 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:02.144636 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:02.147678 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 322/660] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:13:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:02.148900 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34afcd-46db-443a-abb3-a09480493c20 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:02.149230 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34afcd-46db-443a-abb3-a09480493c20 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:02.149489 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34afcd-46db-443a-abb3-a09480493c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:02.171210 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-2c34afcd-46db-443a-abb3-a09480493c20 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:13:02.172706 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2c34afcd-46db-443a-abb3-a09480493c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:13:02.308738 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 323/661] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:13:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:13:02.314100 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4408ef1-bf65-4616-8bf5-ee44a86be3f9 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:02.314290 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c34afcd-46db-443a-abb3-a09480493c20 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:02.349597 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:02.396797 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e729f31-ec88-45fa-b11f-563a53544a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:02.397882 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e729f31-ec88-45fa-b11f-563a53544a63 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:02.398327 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e729f31-ec88-45fa-b11f-563a53544a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:02.398723 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e729f31-ec88-45fa-b11f-563a53544a63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:02.400249 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e729f31-ec88-45fa-b11f-563a53544a63 None admin] RBAC: Adding query filter params (role.id=b9dbf55e6f2947158c1d946f6ee9f907, scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb, user.id=e7da4bf6f3be47b4aa25c6240cb4be03) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:02.400758 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e729f31-ec88-45fa-b11f-563a53544a63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:02.401163 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:02.403981 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e729f31-ec88-45fa-b11f-563a53544a63 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:02.421685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 339/662] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:13:02 2022] GET /identity/v3/role_assignments?user.id=e7da4bf6f3be47b4aa25c6240cb4be03&scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb&role.id=b9dbf55e6f2947158c1d946f6ee9f907 => generated 608 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:02.421685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e729f31-ec88-45fa-b11f-563a53544a63 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:02.488753 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c34afcd-46db-443a-abb3-a09480493c20 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:02.566894 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a930219d-3a03-4768-ae41-d157206d847d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:02.568385 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a930219d-3a03-4768-ae41-d157206d847d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:02.568785 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a930219d-3a03-4768-ae41-d157206d847d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:02.569237 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a930219d-3a03-4768-ae41-d157206d847d None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:02.571326 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a930219d-3a03-4768-ae41-d157206d847d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:02.571862 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a930219d-3a03-4768-ae41-d157206d847d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:02.575926 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a930219d-3a03-4768-ae41-d157206d847d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:02.611636 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 324/663] 10.222.0.87 () {56 vars in 1137 bytes} [Tue Jun 21 18:13:02 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:02.613137 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a930219d-3a03-4768-ae41-d157206d847d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:02.635894 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e729f31-ec88-45fa-b11f-563a53544a63 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:02.686509 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdecfbb-9f7e-4c06-9f0f-ca391da1c6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:02.687219 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdecfbb-9f7e-4c06-9f0f-ca391da1c6d0 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:02.687506 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdecfbb-9f7e-4c06-9f0f-ca391da1c6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:02.687790 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdecfbb-9f7e-4c06-9f0f-ca391da1c6d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:02.688546 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdecfbb-9f7e-4c06-9f0f-ca391da1c6d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:02.688860 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdecfbb-9f7e-4c06-9f0f-ca391da1c6d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:02.690447 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdecfbb-9f7e-4c06-9f0f-ca391da1c6d0 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:02.708325 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 340/664] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:13:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:02.709781 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdecfbb-9f7e-4c06-9f0f-ca391da1c6d0 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:02.791765 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:02.832942 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a930219d-3a03-4768-ae41-d157206d847d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:02.890413 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:02.916465 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf9b474-d6bd-4824-8e18-7ce7386c29df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:02.918881 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9b474-d6bd-4824-8e18-7ce7386c29df None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:02.919927 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9b474-d6bd-4824-8e18-7ce7386c29df None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:02.920682 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9b474-d6bd-4824-8e18-7ce7386c29df None admin] PATH_INFO: `/v3/roles/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:02.945608 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf9b474-d6bd-4824-8e18-7ce7386c29df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:13:02.955651 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdecfbb-9f7e-4c06-9f0f-ca391da1c6d0 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:02.964457 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-0bf9b474-d6bd-4824-8e18-7ce7386c29df None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:13:02.964457 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:02.964457 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:02.964457 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:02.964457 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:02.964457 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:02.964457 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:02.964457 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:02.964457 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:02.964457 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:02.964457 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:02.964457 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:02.964457 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:13:02.964457 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:13:02.964457 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:13:02.964457 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:13:02.966869 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:13:02.966869 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:13:02.966869 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:13:02.966869 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:13:02.966869 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:02.966869 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:02.966869 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:02.966869 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:02.966869 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:02.966869 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:02.966869 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:02.966869 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:02.966869 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:02.966869 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:02.966869 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:02.966869 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:02.969540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:02.969540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:02.969540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:02.969540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:02.969540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:13:02.969540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:13:02.969540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:13:02.969540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:13:02.969540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:13:02.969540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:13:02.969540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:13:02.969540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:02.982440 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 325/665] 10.222.0.87 () {56 vars in 1102 bytes} [Tue Jun 21 18:13:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:02.984656 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf9b474-d6bd-4824-8e18-7ce7386c29df None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:03.045796 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e2f3ce-233c-4fc8-9d18-d5c8ce0fabbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:03.047174 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2f3ce-233c-4fc8-9d18-d5c8ce0fabbe None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:03.048046 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2f3ce-233c-4fc8-9d18-d5c8ce0fabbe None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:03.048807 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2f3ce-233c-4fc8-9d18-d5c8ce0fabbe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:03.218182 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf9b474-d6bd-4824-8e18-7ce7386c29df None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:03.231170 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e2f3ce-233c-4fc8-9d18-d5c8ce0fabbe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:03.234256 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e2f3ce-233c-4fc8-9d18-d5c8ce0fabbe None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:03.289173 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff3d4f7-69ff-406f-b5e6-f8121d2c805c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:03.291279 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff3d4f7-69ff-406f-b5e6-f8121d2c805c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:03.292294 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff3d4f7-69ff-406f-b5e6-f8121d2c805c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:03.293351 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff3d4f7-69ff-406f-b5e6-f8121d2c805c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:03.310279 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff3d4f7-69ff-406f-b5e6-f8121d2c805c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:03.310812 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:03.314843 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff3d4f7-69ff-406f-b5e6-f8121d2c805c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:03.344547 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-4ff3d4f7-69ff-406f-b5e6-f8121d2c805c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 21 18:13:03.344547 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:03.344547 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:03.344547 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:03.344547 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:03.344547 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:03.344547 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:03.344547 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:03.344547 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:03.344547 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:03.344547 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:03.344547 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:03.344547 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:13:03.344547 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:13:03.344547 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:13:03.344547 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:13:03.346604 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:03.346604 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:03.346604 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:13:03.346604 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:13:03.346604 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:13:03.346604 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:13:03.346604 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:03.346604 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:03.346604 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:03.346604 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:03.346604 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:03.346604 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:03.346604 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:03.346604 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:03.346604 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:03.346604 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:03.346604 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:03.346604 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:03.348418 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:03.348418 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:03.348418 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:13:03.348418 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:13:03.348418 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:13:03.348418 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:13:03.348418 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:13:03.348418 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:13:03.348418 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 21 18:13:03.348418 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:03.348418 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 326/666] 10.222.0.87 () {56 vars in 1102 bytes} [Tue Jun 21 18:13:02 2022] GET /identity/v3/users/neutron => generated 85 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:03.350794 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff3d4f7-69ff-406f-b5e6-f8121d2c805c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:03.453268 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:03.469199 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 341/667] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:13:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 760 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:13:03.471619 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e2f3ce-233c-4fc8-9d18-d5c8ce0fabbe None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:03.583163 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff3d4f7-69ff-406f-b5e6-f8121d2c805c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:03.650617 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9119272b-1186-4364-9992-9571c95d95fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:03.651871 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119272b-1186-4364-9992-9571c95d95fb None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:03.652346 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119272b-1186-4364-9992-9571c95d95fb None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:03.652893 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119272b-1186-4364-9992-9571c95d95fb None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:03.654332 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9119272b-1186-4364-9992-9571c95d95fb None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:03.654856 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9119272b-1186-4364-9992-9571c95d95fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:03.657374 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9119272b-1186-4364-9992-9571c95d95fb None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:03.681465 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 327/668] 10.222.0.87 () {56 vars in 1103 bytes} [Tue Jun 21 18:13:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:03.681465 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9119272b-1186-4364-9992-9571c95d95fb None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:03.694513 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e2f3ce-233c-4fc8-9d18-d5c8ce0fabbe None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:03.755009 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b40b6-98ed-4d93-bfb9-f8244d6f7916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:03.756039 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b40b6-98ed-4d93-bfb9-f8244d6f7916 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:03.756534 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b40b6-98ed-4d93-bfb9-f8244d6f7916 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:03.757089 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b40b6-98ed-4d93-bfb9-f8244d6f7916 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:03.758261 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239b40b6-98ed-4d93-bfb9-f8244d6f7916 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:03.758822 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239b40b6-98ed-4d93-bfb9-f8244d6f7916 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:03.761536 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239b40b6-98ed-4d93-bfb9-f8244d6f7916 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:03.766765 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:03.792266 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 342/669] 10.222.0.87 () {56 vars in 1139 bytes} [Tue Jun 21 18:13:03 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:03.793561 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b40b6-98ed-4d93-bfb9-f8244d6f7916 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:03.902737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9119272b-1186-4364-9992-9571c95d95fb None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:03.909239 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:03.956327 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b40b6-98ed-4d93-bfb9-f8244d6f7916 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:03.983806 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d759cf8-2ba4-40ae-b0c4-df589b748d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:03.985348 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d759cf8-2ba4-40ae-b0c4-df589b748d5d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:03.985860 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d759cf8-2ba4-40ae-b0c4-df589b748d5d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:03.986310 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d759cf8-2ba4-40ae-b0c4-df589b748d5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:04.002032 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d759cf8-2ba4-40ae-b0c4-df589b748d5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:04.004222 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d759cf8-2ba4-40ae-b0c4-df589b748d5d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:04.030356 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba740a6-882a-4b0e-bbf4-1cfc21b8394a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:04.032358 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba740a6-882a-4b0e-bbf4-1cfc21b8394a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:04.032358 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba740a6-882a-4b0e-bbf4-1cfc21b8394a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:04.032983 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba740a6-882a-4b0e-bbf4-1cfc21b8394a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:04.034795 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-0d759cf8-2ba4-40ae-b0c4-df589b748d5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:04.034795 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:04.034795 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:04.034795 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:04.034795 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:04.034795 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:04.034795 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:04.034795 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:04.034795 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:04.034795 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:04.034795 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:04.034795 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:04.034795 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:13:04.034795 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:13:04.034795 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:13:04.035795 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:13:04.035795 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:04.035795 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:04.035795 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:04.035795 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:04.035795 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:04.035795 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:04.035795 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:04.035795 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:04.035795 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:04.035795 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:04.035795 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:04.035795 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:04.035795 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:04.035795 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:04.035795 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:04.035795 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:04.039511 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:13:04.039511 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:13:04.039511 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:04.039511 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:04.043997 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 328/670] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:13:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:04.045307 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d759cf8-2ba4-40ae-b0c4-df589b748d5d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:04.057018 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba740a6-882a-4b0e-bbf4-1cfc21b8394a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:04.059509 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba740a6-882a-4b0e-bbf4-1cfc21b8394a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:04.085423 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-fba740a6-882a-4b0e-bbf4-1cfc21b8394a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:04.085423 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:04.085423 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:04.085423 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:04.085423 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:04.085423 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:04.085423 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:04.085423 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:04.085423 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:04.085423 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:04.085423 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:04.085423 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:04.085423 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:13:04.085423 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:13:04.085423 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:13:04.086536 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:13:04.086536 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:04.086536 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:04.086536 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:04.086536 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:04.086536 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:04.086536 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:04.086536 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:04.086536 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:04.086536 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:04.086536 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:04.086536 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:04.086536 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:04.086536 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:04.086536 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:04.086536 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:04.086536 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:04.087470 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:13:04.087470 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:13:04.087470 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:04.087470 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:04.087695 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 343/671] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:13:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:04.089827 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba740a6-882a-4b0e-bbf4-1cfc21b8394a None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:04.161385 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:04.242193 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba740a6-882a-4b0e-bbf4-1cfc21b8394a None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:04.262089 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d759cf8-2ba4-40ae-b0c4-df589b748d5d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:04.294259 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379518e-a6b1-49c3-a986-5f40015dbdd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:04.295345 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379518e-a6b1-49c3-a986-5f40015dbdd2 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:04.295888 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379518e-a6b1-49c3-a986-5f40015dbdd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:04.296326 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379518e-a6b1-49c3-a986-5f40015dbdd2 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:04.297251 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0379518e-a6b1-49c3-a986-5f40015dbdd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:04.297744 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0379518e-a6b1-49c3-a986-5f40015dbdd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:04.299339 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0379518e-a6b1-49c3-a986-5f40015dbdd2 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:04.314240 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 344/672] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:13:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:04.315460 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379518e-a6b1-49c3-a986-5f40015dbdd2 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:04.336482 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2035b452-7d4f-403f-8db8-e9378fc57aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:04.337665 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035b452-7d4f-403f-8db8-e9378fc57aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:04.338537 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035b452-7d4f-403f-8db8-e9378fc57aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:04.338819 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035b452-7d4f-403f-8db8-e9378fc57aa6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:04.392807 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:04.460465 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379518e-a6b1-49c3-a986-5f40015dbdd2 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:04.471127 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:04.486678 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2035b452-7d4f-403f-8db8-e9378fc57aa6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:04.489702 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2035b452-7d4f-403f-8db8-e9378fc57aa6 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:04.515317 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf42035-eff4-4935-b7e7-a586864bf8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:04.516060 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf42035-eff4-4935-b7e7-a586864bf8dd None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:04.516352 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf42035-eff4-4935-b7e7-a586864bf8dd None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:04.516706 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf42035-eff4-4935-b7e7-a586864bf8dd None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:04.517617 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf42035-eff4-4935-b7e7-a586864bf8dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:04.518227 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf42035-eff4-4935-b7e7-a586864bf8dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:04.520123 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf42035-eff4-4935-b7e7-a586864bf8dd None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:04.541007 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 345/673] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:13:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:04.542295 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf42035-eff4-4935-b7e7-a586864bf8dd None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:04.681565 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf42035-eff4-4935-b7e7-a586864bf8dd None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:04.698761 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 329/674] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:13:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 654 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:13:04.700911 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2035b452-7d4f-403f-8db8-e9378fc57aa6 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:04.730363 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b14ec8-1fff-4841-991b-74269fd5afa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:04.731179 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b14ec8-1fff-4841-991b-74269fd5afa9 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:04.731476 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b14ec8-1fff-4841-991b-74269fd5afa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:04.731868 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b14ec8-1fff-4841-991b-74269fd5afa9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:04.741197 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b14ec8-1fff-4841-991b-74269fd5afa9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:04.742717 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b14ec8-1fff-4841-991b-74269fd5afa9 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:04.763490 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-19b14ec8-1fff-4841-991b-74269fd5afa9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:13:04.763490 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:04.763490 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:04.763490 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:04.763490 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:04.763490 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:04.763490 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:04.763490 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:04.763490 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:04.763490 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:04.763490 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:04.763490 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:04.763490 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:13:04.763490 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:13:04.763490 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:13:04.765607 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:13:04.765607 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:04.765607 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:04.765607 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:04.765607 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:04.765607 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:04.765607 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:04.765607 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:04.765607 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:04.765607 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:04.765607 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:04.765607 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:04.765607 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:04.765607 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:04.765607 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:04.765607 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:04.765607 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:04.767811 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:13:04.767811 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:13:04.767811 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:13:04.767811 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:13:04.767811 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:13:04.767811 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:13:04.767811 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:13:04.767811 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:04.767811 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 346/675] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:13:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:04.767811 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b14ec8-1fff-4841-991b-74269fd5afa9 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:04.806179 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:04.834306 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:04.940852 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2035b452-7d4f-403f-8db8-e9378fc57aa6 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:04.949262 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b14ec8-1fff-4841-991b-74269fd5afa9 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:05.000013 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758f63c-64e0-4784-a3d8-8109f15e5cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:05.001370 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758f63c-64e0-4784-a3d8-8109f15e5cfa None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:05.001890 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758f63c-64e0-4784-a3d8-8109f15e5cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:05.002396 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758f63c-64e0-4784-a3d8-8109f15e5cfa None admin] PATH_INFO: `/v3/users/swift` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:05.016593 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e440ff9-a761-4707-941b-c4837413f325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:05.018386 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e440ff9-a761-4707-941b-c4837413f325 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:05.018626 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8758f63c-64e0-4784-a3d8-8109f15e5cfa None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:05.018910 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e440ff9-a761-4707-941b-c4837413f325 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:05.019275 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e440ff9-a761-4707-941b-c4837413f325 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:05.021184 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8758f63c-64e0-4784-a3d8-8109f15e5cfa None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:05.040670 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e440ff9-a761-4707-941b-c4837413f325 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:05.043934 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e440ff9-a761-4707-941b-c4837413f325 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:05.050717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-8758f63c-64e0-4784-a3d8-8109f15e5cfa None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 21 18:13:05.050717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:05.050717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:05.050717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:05.050717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:05.050717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:05.050717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:05.050717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:05.050717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:05.050717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:05.050717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:05.050717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:05.050717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:13:05.050717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:13:05.050717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:13:05.050717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:13:05.052526 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:05.052526 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:05.052526 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:13:05.052526 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:13:05.052526 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:13:05.052526 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:13:05.052526 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:05.052526 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:05.052526 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:05.052526 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:05.052526 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:05.052526 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:05.052526 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:05.052526 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:05.052526 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:05.052526 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:05.052526 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:05.052526 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:05.060961 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:05.060961 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:05.060961 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:13:05.060961 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:13:05.060961 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:13:05.060961 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:13:05.060961 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:13:05.060961 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:13:05.060961 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 21 18:13:05.060961 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:05.060961 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 330/676] 10.222.0.87 () {56 vars in 1096 bytes} [Tue Jun 21 18:13:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:05.060961 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758f63c-64e0-4784-a3d8-8109f15e5cfa None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:05.078636 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-6e440ff9-a761-4707-941b-c4837413f325 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:05.078636 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:05.078636 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:05.078636 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:05.078636 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:05.078636 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:05.078636 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:05.078636 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:05.078636 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:05.078636 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:05.078636 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:05.078636 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:05.078636 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:13:05.078636 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:13:05.078636 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:13:05.079649 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:13:05.079649 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:05.079649 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:05.079649 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:05.079649 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:05.079649 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:05.079649 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:05.079649 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:05.079649 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:05.079649 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:05.079649 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:05.079649 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:05.079649 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:05.079649 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:05.079649 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:05.079649 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:05.079649 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:05.080546 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:13:05.080546 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:13:05.080546 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:05.080546 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:05.080546 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 347/677] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:13:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:05.082736 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d0642-d5bf-49a4-8329-7e3a7309aae7 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:05.083216 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d0642-d5bf-49a4-8329-7e3a7309aae7 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:05.083549 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d0642-d5bf-49a4-8329-7e3a7309aae7 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:05.087838 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 348/678] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:13:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:13:05.087838 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504d0642-d5bf-49a4-8329-7e3a7309aae7 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:05.232784 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:05.270826 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758f63c-64e0-4784-a3d8-8109f15e5cfa None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:05.279838 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504d0642-d5bf-49a4-8329-7e3a7309aae7 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:05.291795 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:05.316827 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5c1af-7076-4669-95e6-a2ac6e199818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:05.316827 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5c1af-7076-4669-95e6-a2ac6e199818 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:05.318769 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5c1af-7076-4669-95e6-a2ac6e199818 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:05.318769 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5c1af-7076-4669-95e6-a2ac6e199818 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:05.319392 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f5c1af-7076-4669-95e6-a2ac6e199818 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:05.319785 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f5c1af-7076-4669-95e6-a2ac6e199818 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:05.321977 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f5c1af-7076-4669-95e6-a2ac6e199818 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:05.353593 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 331/679] 10.222.0.87 () {56 vars in 1148 bytes} [Tue Jun 21 18:13:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:05.353593 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5c1af-7076-4669-95e6-a2ac6e199818 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:05.358973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b984d146-f6e7-4d9f-8aef-caf7a7d20a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:05.360398 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b984d146-f6e7-4d9f-8aef-caf7a7d20a98 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:05.360892 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b984d146-f6e7-4d9f-8aef-caf7a7d20a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:05.361400 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b984d146-f6e7-4d9f-8aef-caf7a7d20a98 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:05.363173 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b984d146-f6e7-4d9f-8aef-caf7a7d20a98 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:05.363714 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b984d146-f6e7-4d9f-8aef-caf7a7d20a98 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:05.366615 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b984d146-f6e7-4d9f-8aef-caf7a7d20a98 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:05.406218 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 349/680] 10.222.0.87 () {56 vars in 1135 bytes} [Tue Jun 21 18:13:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:05.408669 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d53f31-beed-415a-b974-cc96556b4803 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:05.409171 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d53f31-beed-415a-b974-cc96556b4803 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:05.409716 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d53f31-beed-415a-b974-cc96556b4803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:05.441098 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-f2d53f31-beed-415a-b974-cc96556b4803 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:13:05.443300 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f2d53f31-beed-415a-b974-cc96556b4803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:13:05.565351 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5c1af-7076-4669-95e6-a2ac6e199818 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:05.584210 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 350/681] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:13:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:13:05.585584 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d53f31-beed-415a-b974-cc96556b4803 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:05.624586 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc878ce-8505-4735-bab1-a9b9e3af3ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:05.625276 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc878ce-8505-4735-bab1-a9b9e3af3ddf None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:05.625561 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc878ce-8505-4735-bab1-a9b9e3af3ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:05.625886 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc878ce-8505-4735-bab1-a9b9e3af3ddf None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:05.626625 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc878ce-8505-4735-bab1-a9b9e3af3ddf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:05.626932 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc878ce-8505-4735-bab1-a9b9e3af3ddf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:05.628356 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc878ce-8505-4735-bab1-a9b9e3af3ddf None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:05.645695 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 332/682] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:13:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:05.645695 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc878ce-8505-4735-bab1-a9b9e3af3ddf None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:05.705102 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:05.706073 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:05.756046 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d53f31-beed-415a-b974-cc96556b4803 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:05.810903 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc878ce-8505-4735-bab1-a9b9e3af3ddf None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:05.817150 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e75e42-ddda-4a49-b305-d4606aceab42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:05.818263 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e75e42-ddda-4a49-b305-d4606aceab42 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:05.818751 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e75e42-ddda-4a49-b305-d4606aceab42 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:05.819227 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e75e42-ddda-4a49-b305-d4606aceab42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:05.820862 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e75e42-ddda-4a49-b305-d4606aceab42 None admin] RBAC: Adding query filter params (role.id=b9dbf55e6f2947158c1d946f6ee9f907, scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb, user.id=b683e6f78abd4986b415eb9ddf34cb8d) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:05.821415 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e75e42-ddda-4a49-b305-d4606aceab42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:05.824272 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e75e42-ddda-4a49-b305-d4606aceab42 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:05.836306 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 351/683] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:13:05 2022] GET /identity/v3/role_assignments?user.id=b683e6f78abd4986b415eb9ddf34cb8d&scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb&role.id=b9dbf55e6f2947158c1d946f6ee9f907 => generated 608 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:05.838194 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e75e42-ddda-4a49-b305-d4606aceab42 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:05.888470 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751aba86-966a-4b97-966d-78f2d392fb26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:05.889760 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751aba86-966a-4b97-966d-78f2d392fb26 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:05.890250 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751aba86-966a-4b97-966d-78f2d392fb26 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:05.890725 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751aba86-966a-4b97-966d-78f2d392fb26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:06.029508 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751aba86-966a-4b97-966d-78f2d392fb26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:06.031306 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751aba86-966a-4b97-966d-78f2d392fb26 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:06.061109 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e75e42-ddda-4a49-b305-d4606aceab42 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:06.084384 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:06.140492 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:06.155663 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9647357-0f4b-4cc5-87d4-0a63a4ebfa9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:06.156695 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9647357-0f4b-4cc5-87d4-0a63a4ebfa9f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:06.157161 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9647357-0f4b-4cc5-87d4-0a63a4ebfa9f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:06.157686 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9647357-0f4b-4cc5-87d4-0a63a4ebfa9f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:06.181082 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9647357-0f4b-4cc5-87d4-0a63a4ebfa9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:13:06.194272 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-f9647357-0f4b-4cc5-87d4-0a63a4ebfa9f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:13:06.194272 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:06.194272 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:06.194272 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:06.194272 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:06.194272 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:06.194272 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:06.194272 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:06.194272 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:06.194272 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:06.194272 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:06.194272 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:06.194272 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:13:06.194272 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:13:06.194272 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:13:06.194272 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:13:06.196382 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:13:06.196382 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:13:06.196382 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:13:06.196382 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:13:06.196382 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:06.196382 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:06.196382 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:06.196382 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:06.196382 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:06.196382 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:06.196382 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:06.196382 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:06.196382 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:06.196382 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:06.196382 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:06.196382 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:06.198389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:06.198389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:06.198389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:06.198389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:06.198389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:13:06.198389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:13:06.198389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:13:06.198389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:13:06.198389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:13:06.198389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:13:06.198389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:13:06.198389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:06.198389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 352/684] 10.222.0.87 () {56 vars in 1096 bytes} [Tue Jun 21 18:13:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:06.201960 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9647357-0f4b-4cc5-87d4-0a63a4ebfa9f None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:06.215338 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 333/685] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:13:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:13:06.217689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f51a40c-8f8b-462c-9baa-b590abd6989f None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:06.218236 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f51a40c-8f8b-462c-9baa-b590abd6989f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:06.218708 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f51a40c-8f8b-462c-9baa-b590abd6989f None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:06.220574 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 334/686] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:13:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:13:06.221729 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f51a40c-8f8b-462c-9baa-b590abd6989f None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:06.404228 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9647357-0f4b-4cc5-87d4-0a63a4ebfa9f None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:06.412713 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f51a40c-8f8b-462c-9baa-b590abd6989f None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:06.460642 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d79e15-acc0-4448-84be-4151691d58c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:06.462106 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d79e15-acc0-4448-84be-4151691d58c8 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:06.462602 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d79e15-acc0-4448-84be-4151691d58c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:06.463205 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d79e15-acc0-4448-84be-4151691d58c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:06.475769 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d99d3ea-db25-44ec-b663-f5f6a0f4e543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:06.476970 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99d3ea-db25-44ec-b663-f5f6a0f4e543 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:06.479252 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99d3ea-db25-44ec-b663-f5f6a0f4e543 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:06.479252 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99d3ea-db25-44ec-b663-f5f6a0f4e543 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:06.479466 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d79e15-acc0-4448-84be-4151691d58c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:06.480049 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d99d3ea-db25-44ec-b663-f5f6a0f4e543 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:06.480550 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d99d3ea-db25-44ec-b663-f5f6a0f4e543 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:06.481873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d79e15-acc0-4448-84be-4151691d58c8 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:06.483075 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d99d3ea-db25-44ec-b663-f5f6a0f4e543 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:06.488836 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:06.504258 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 335/687] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:13:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:06.506789 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b0ce4-5c78-4048-b0b2-5078bec981b3 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:06.507279 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b0ce4-5c78-4048-b0b2-5078bec981b3 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:06.507740 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b0ce4-5c78-4048-b0b2-5078bec981b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:06.514821 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-39d79e15-acc0-4448-84be-4151691d58c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:13:06.514821 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:06.514821 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:06.514821 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:06.514821 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:06.514821 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:06.514821 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:06.514821 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:06.514821 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:06.514821 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:06.514821 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:06.514821 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:06.514821 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:13:06.514821 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:13:06.514821 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:13:06.516511 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:13:06.516511 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:06.516511 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:06.516511 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:06.516511 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:06.516511 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:06.516511 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:06.516511 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:06.516511 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:06.516511 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:06.516511 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:06.516511 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:06.516511 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:06.516511 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:06.516511 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:06.516511 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:06.516511 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:06.518210 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:13:06.518210 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:13:06.518210 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:13:06.518210 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:13:06.518210 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:13:06.518210 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:13:06.518210 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:13:06.518210 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:06.519922 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 353/688] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:13:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:06.528146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d79e15-acc0-4448-84be-4151691d58c8 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:06.535434 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-0f2b0ce4-5c78-4048-b0b2-5078bec981b3 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:13:06.537505 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0f2b0ce4-5c78-4048-b0b2-5078bec981b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:13:06.608495 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:06.670829 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 336/689] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:13:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:13:06.670829 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2b0ce4-5c78-4048-b0b2-5078bec981b3 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:06.694425 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d79e15-acc0-4448-84be-4151691d58c8 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:06.750896 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d297a6-2450-4769-82d4-59a715f02913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:06.751720 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d297a6-2450-4769-82d4-59a715f02913 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:06.752004 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d297a6-2450-4769-82d4-59a715f02913 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:06.752274 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d297a6-2450-4769-82d4-59a715f02913 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:06.765871 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d297a6-2450-4769-82d4-59a715f02913 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:06.768060 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d297a6-2450-4769-82d4-59a715f02913 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:06.794639 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-d4d297a6-2450-4769-82d4-59a715f02913 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:06.794639 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:06.794639 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:06.794639 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:06.794639 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:06.794639 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:06.794639 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:06.794639 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:06.794639 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:06.794639 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:06.794639 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:06.794639 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:06.794639 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:13:06.794639 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:13:06.794639 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:13:06.796440 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:13:06.796440 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:06.796440 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:06.796440 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:06.796440 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:06.796440 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:06.796440 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:06.796440 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:06.796440 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:06.796440 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:06.796440 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:06.796440 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:06.796440 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:06.796440 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:06.796440 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:06.796440 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:06.796440 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:06.799063 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:13:06.799063 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:13:06.799063 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:06.799063 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:06.799063 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 354/690] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:13:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:06.799063 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d297a6-2450-4769-82d4-59a715f02913 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:06.895032 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2b0ce4-5c78-4048-b0b2-5078bec981b3 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:06.949294 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:06.965719 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083d3890-805e-41a8-ad27-01d4d49043dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:06.967621 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d3890-805e-41a8-ad27-01d4d49043dd None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:06.967621 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d3890-805e-41a8-ad27-01d4d49043dd None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:06.967954 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d3890-805e-41a8-ad27-01d4d49043dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:06.984666 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083d3890-805e-41a8-ad27-01d4d49043dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:06.986357 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083d3890-805e-41a8-ad27-01d4d49043dd None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:07.006707 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-083d3890-805e-41a8-ad27-01d4d49043dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:07.006707 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:07.006707 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:07.006707 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:07.006707 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:07.006707 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:07.006707 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:07.006707 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:07.006707 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:07.006707 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:07.006707 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:07.006707 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:07.006707 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:13:07.006707 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:13:07.006707 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:13:07.008585 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:13:07.008585 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:07.008585 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:07.008585 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:07.008585 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:07.008585 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:07.008585 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:07.008585 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:07.008585 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:07.008585 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:07.008585 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:07.008585 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:07.008585 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:07.008585 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:07.008585 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:07.008585 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:07.008585 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:07.010219 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:13:07.010219 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:13:07.010219 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:07.010219 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:07.010219 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 337/691] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:13:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:07.010219 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083d3890-805e-41a8-ad27-01d4d49043dd None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:07.023401 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d297a6-2450-4769-82d4-59a715f02913 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:07.028059 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:07.094826 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18abc15-8782-4077-a453-3bf66b120478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:07.095949 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18abc15-8782-4077-a453-3bf66b120478 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:07.096439 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18abc15-8782-4077-a453-3bf66b120478 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:07.096923 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18abc15-8782-4077-a453-3bf66b120478 None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:07.098470 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18abc15-8782-4077-a453-3bf66b120478 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:07.099039 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18abc15-8782-4077-a453-3bf66b120478 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:07.102235 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18abc15-8782-4077-a453-3bf66b120478 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:07.130184 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 355/692] 10.222.0.87 () {56 vars in 1148 bytes} [Tue Jun 21 18:13:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:07.132112 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18abc15-8782-4077-a453-3bf66b120478 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:07.172179 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083d3890-805e-41a8-ad27-01d4d49043dd None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:07.235878 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d08980-7aa8-4865-a15b-1c38e4fd1e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:07.237207 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d08980-7aa8-4865-a15b-1c38e4fd1e68 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:07.237694 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d08980-7aa8-4865-a15b-1c38e4fd1e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:07.238110 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d08980-7aa8-4865-a15b-1c38e4fd1e68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:07.254712 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d08980-7aa8-4865-a15b-1c38e4fd1e68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:07.257766 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d08980-7aa8-4865-a15b-1c38e4fd1e68 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:07.293095 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-a8d08980-7aa8-4865-a15b-1c38e4fd1e68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:07.293095 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:07.293095 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:07.293095 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:07.293095 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:07.293095 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:07.293095 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:07.293095 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:07.293095 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:07.293095 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:07.293095 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:07.293095 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:07.293095 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:13:07.293095 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:13:07.293095 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:13:07.294383 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:13:07.294383 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:07.294383 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:07.294383 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:07.294383 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:07.294383 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:07.294383 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:07.294383 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:07.294383 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:07.294383 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:07.294383 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:07.294383 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:07.294383 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:07.294383 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:07.294383 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:07.294383 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:07.294383 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:07.295298 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:13:07.295298 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:13:07.295298 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:07.295298 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:07.296210 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 338/693] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:13:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:07.297667 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d08980-7aa8-4865-a15b-1c38e4fd1e68 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:07.313735 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18abc15-8782-4077-a453-3bf66b120478 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:07.361518 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b00bc3-b2b0-4a26-a3e3-5c250b4109f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:07.362386 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b00bc3-b2b0-4a26-a3e3-5c250b4109f5 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:07.362985 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b00bc3-b2b0-4a26-a3e3-5c250b4109f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:07.363430 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b00bc3-b2b0-4a26-a3e3-5c250b4109f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:07.364973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b00bc3-b2b0-4a26-a3e3-5c250b4109f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:07.365502 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b00bc3-b2b0-4a26-a3e3-5c250b4109f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:07.365731 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:07.368322 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b00bc3-b2b0-4a26-a3e3-5c250b4109f5 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:07.393999 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 356/694] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:13:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:07.395548 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b00bc3-b2b0-4a26-a3e3-5c250b4109f5 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:07.444806 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:07.483486 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d08980-7aa8-4865-a15b-1c38e4fd1e68 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:07.553827 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67aeb3b-2ed8-42c8-88bf-8d636d50e73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:07.555684 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67aeb3b-2ed8-42c8-88bf-8d636d50e73f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:07.556159 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67aeb3b-2ed8-42c8-88bf-8d636d50e73f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:07.556450 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67aeb3b-2ed8-42c8-88bf-8d636d50e73f None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:07.557537 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67aeb3b-2ed8-42c8-88bf-8d636d50e73f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:07.557881 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67aeb3b-2ed8-42c8-88bf-8d636d50e73f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:07.559329 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67aeb3b-2ed8-42c8-88bf-8d636d50e73f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:07.580981 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 339/695] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:13:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:07.581934 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67aeb3b-2ed8-42c8-88bf-8d636d50e73f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:07.603155 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b00bc3-b2b0-4a26-a3e3-5c250b4109f5 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:07.661824 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5747a6-9c4a-4109-83d4-90b805450b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:07.662849 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5747a6-9c4a-4109-83d4-90b805450b7b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:07.663317 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5747a6-9c4a-4109-83d4-90b805450b7b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:07.663693 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5747a6-9c4a-4109-83d4-90b805450b7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:07.665160 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5747a6-9c4a-4109-83d4-90b805450b7b None admin] RBAC: Adding query filter params (role.id=b9dbf55e6f2947158c1d946f6ee9f907, scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb, user.id=480e2529015c4c2fb4d6cd1985009fde) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:07.665549 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5747a6-9c4a-4109-83d4-90b805450b7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:07.668018 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5747a6-9c4a-4109-83d4-90b805450b7b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:07.681604 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 357/696] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:13:07 2022] GET /identity/v3/role_assignments?user.id=480e2529015c4c2fb4d6cd1985009fde&scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb&role.id=b9dbf55e6f2947158c1d946f6ee9f907 => generated 608 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:07.681604 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5747a6-9c4a-4109-83d4-90b805450b7b None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:07.774665 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:07.793672 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67aeb3b-2ed8-42c8-88bf-8d636d50e73f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:07.848243 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d22cab-768a-47d6-9340-6a08bd0ccd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:07.849342 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d22cab-768a-47d6-9340-6a08bd0ccd79 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:07.850292 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d22cab-768a-47d6-9340-6a08bd0ccd79 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:07.851549 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d22cab-768a-47d6-9340-6a08bd0ccd79 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:07.852671 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d22cab-768a-47d6-9340-6a08bd0ccd79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:07.853230 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d22cab-768a-47d6-9340-6a08bd0ccd79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:07.855632 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d22cab-768a-47d6-9340-6a08bd0ccd79 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:07.867945 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:07.892181 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 340/697] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:13:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:07.892181 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d22cab-768a-47d6-9340-6a08bd0ccd79 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:07.905654 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5747a6-9c4a-4109-83d4-90b805450b7b None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:07.970013 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e77efc6-7dc7-4f4f-994c-edf8f9c37f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:07.971606 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77efc6-7dc7-4f4f-994c-edf8f9c37f4d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:07.971759 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77efc6-7dc7-4f4f-994c-edf8f9c37f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:07.972140 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77efc6-7dc7-4f4f-994c-edf8f9c37f4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:07.990831 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e77efc6-7dc7-4f4f-994c-edf8f9c37f4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:07.994630 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e77efc6-7dc7-4f4f-994c-edf8f9c37f4d None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:08.016216 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-2e77efc6-7dc7-4f4f-994c-edf8f9c37f4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:13:08.016216 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:08.016216 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:08.016216 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:08.016216 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:08.016216 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:08.016216 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:08.016216 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:08.016216 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:08.016216 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:08.016216 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:08.016216 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:08.016216 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:13:08.016216 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:13:08.016216 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:13:08.018223 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:13:08.018223 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:08.018223 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:08.018223 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:08.018223 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:08.018223 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:08.018223 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:08.018223 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:08.018223 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:08.018223 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:08.018223 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:08.018223 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:08.018223 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:08.018223 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:08.018223 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:08.018223 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:08.018223 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:08.020220 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:13:08.020220 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:13:08.020220 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:13:08.020220 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:13:08.020220 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:13:08.020220 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:13:08.020220 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:13:08.020220 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:08.025756 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 358/698] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:13:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:08.025756 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e77efc6-7dc7-4f4f-994c-edf8f9c37f4d None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:08.099567 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d22cab-768a-47d6-9340-6a08bd0ccd79 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:08.147546 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b447f8e5-f8b1-439d-abdf-c7aa2e410da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:08.148912 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447f8e5-f8b1-439d-abdf-c7aa2e410da6 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:08.149327 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447f8e5-f8b1-439d-abdf-c7aa2e410da6 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:08.149762 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447f8e5-f8b1-439d-abdf-c7aa2e410da6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:08.159975 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b447f8e5-f8b1-439d-abdf-c7aa2e410da6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:08.161671 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b447f8e5-f8b1-439d-abdf-c7aa2e410da6 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:08.183116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-b447f8e5-f8b1-439d-abdf-c7aa2e410da6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:13:08.183116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:08.183116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:08.183116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:08.183116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:08.183116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:08.183116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:08.183116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:08.183116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:08.183116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:08.183116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:08.183116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:08.183116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:13:08.183116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:13:08.183116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:13:08.183116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:13:08.185116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:08.185116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:08.185116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:13:08.185116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:13:08.185116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:13:08.185116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:13:08.185116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:08.185116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:08.185116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:08.185116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:08.185116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:08.185116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:08.185116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:08.185116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:08.185116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:08.185116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:08.185116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:08.185116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:08.187225 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:08.187225 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:08.187225 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:13:08.187225 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:13:08.187225 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:13:08.187225 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:13:08.187225 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:13:08.187225 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:13:08.187225 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:13:08.187225 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:08.189754 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 341/699] 10.222.0.87 () {56 vars in 1093 bytes} [Tue Jun 21 18:13:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:08.191639 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b447f8e5-f8b1-439d-abdf-c7aa2e410da6 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:08.215360 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:08.246489 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e77efc6-7dc7-4f4f-994c-edf8f9c37f4d None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:08.308797 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7ce564-7515-4fa1-9e91-bf4d0a6fec6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:08.310227 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ce564-7515-4fa1-9e91-bf4d0a6fec6a None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:08.310844 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ce564-7515-4fa1-9e91-bf4d0a6fec6a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:08.312775 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ce564-7515-4fa1-9e91-bf4d0a6fec6a None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:08.314991 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7ce564-7515-4fa1-9e91-bf4d0a6fec6a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:08.320094 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7ce564-7515-4fa1-9e91-bf4d0a6fec6a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:08.371168 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 359/700] 10.222.0.87 () {60 vars in 1132 bytes} [Tue Jun 21 18:13:08 2022] POST /identity/v3/users => generated 254 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:13:08.374082 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7ce564-7515-4fa1-9e91-bf4d0a6fec6a None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:08.381745 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:08.422080 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b447f8e5-f8b1-439d-abdf-c7aa2e410da6 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:08.488269 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9e3006-4558-4bf2-9fca-576b5b836cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:08.490738 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e3006-4558-4bf2-9fca-576b5b836cfb None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:08.491554 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e3006-4558-4bf2-9fca-576b5b836cfb None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:08.492251 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e3006-4558-4bf2-9fca-576b5b836cfb None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:08.494069 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9e3006-4558-4bf2-9fca-576b5b836cfb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:08.494938 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9e3006-4558-4bf2-9fca-576b5b836cfb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:08.497407 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9e3006-4558-4bf2-9fca-576b5b836cfb None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:08.514279 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 342/701] 10.222.0.87 () {56 vars in 1148 bytes} [Tue Jun 21 18:13:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:08.521306 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9e3006-4558-4bf2-9fca-576b5b836cfb None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:08.615595 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7ce564-7515-4fa1-9e91-bf4d0a6fec6a None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:08.616083 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:08.656156 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9e3006-4558-4bf2-9fca-576b5b836cfb None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:08.682138 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e74ac4-48ad-4e97-bad4-50aa53196ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:08.683410 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e74ac4-48ad-4e97-bad4-50aa53196ba8 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:08.683874 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e74ac4-48ad-4e97-bad4-50aa53196ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:08.684465 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e74ac4-48ad-4e97-bad4-50aa53196ba8 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:08.686121 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e74ac4-48ad-4e97-bad4-50aa53196ba8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:08.686676 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e74ac4-48ad-4e97-bad4-50aa53196ba8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:08.689514 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e74ac4-48ad-4e97-bad4-50aa53196ba8 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:08.701687 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8cfa81-77a8-49c4-9b25-863a7ef43701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:08.703325 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8cfa81-77a8-49c4-9b25-863a7ef43701 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:08.703938 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8cfa81-77a8-49c4-9b25-863a7ef43701 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:08.704679 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8cfa81-77a8-49c4-9b25-863a7ef43701 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:08.706059 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8cfa81-77a8-49c4-9b25-863a7ef43701 None admin] RBAC: Adding query filter params (role.id=b9dbf55e6f2947158c1d946f6ee9f907, scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb, user.id=515b3571045e4fe29565307c8ecc10b2) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:08.706616 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8cfa81-77a8-49c4-9b25-863a7ef43701 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:08.708663 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8cfa81-77a8-49c4-9b25-863a7ef43701 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:08.717525 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 343/702] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:13:08 2022] GET /identity/v3/role_assignments?user.id=515b3571045e4fe29565307c8ecc10b2&scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb&role.id=b9dbf55e6f2947158c1d946f6ee9f907 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:13:08.734598 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 360/703] 10.222.0.87 () {56 vars in 1133 bytes} [Tue Jun 21 18:13:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:08.743416 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8cfa81-77a8-49c4-9b25-863a7ef43701 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:08.914013 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8cfa81-77a8-49c4-9b25-863a7ef43701 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:08.929788 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ce988-bbb0-45d7-9b06-b24cbc8f4281 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:08.930570 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ce988-bbb0-45d7-9b06-b24cbc8f4281 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:08.930570 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ce988-bbb0-45d7-9b06-b24cbc8f4281 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:08.937020 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 361/704] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:13:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:13:08.940007 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef4330-56c0-4241-bcaf-39e46416e2f2 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:08.940456 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef4330-56c0-4241-bcaf-39e46416e2f2 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:08.940840 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef4330-56c0-4241-bcaf-39e46416e2f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:08.966206 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-d4ef4330-56c0-4241-bcaf-39e46416e2f2 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:13:08.967998 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d4ef4330-56c0-4241-bcaf-39e46416e2f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:13:08.977175 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b02e30-5526-47be-a3f6-2ff1c59320a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:08.977996 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b02e30-5526-47be-a3f6-2ff1c59320a9 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:08.978335 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b02e30-5526-47be-a3f6-2ff1c59320a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:08.978627 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b02e30-5526-47be-a3f6-2ff1c59320a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:08.983134 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:09.055485 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:09.075109 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b02e30-5526-47be-a3f6-2ff1c59320a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:09.076279 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 362/705] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:13:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:13:09.082483 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b02e30-5526-47be-a3f6-2ff1c59320a9 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:09.102820 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ef4330-56c0-4241-bcaf-39e46416e2f2 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:09.212081 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 344/706] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:13:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 21 18:13:09.223700 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b02e30-5526-47be-a3f6-2ff1c59320a9 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:09.293745 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:09.314804 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ef4330-56c0-4241-bcaf-39e46416e2f2 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:09.413680 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50040e05-a8d1-4212-9ccc-a83093d44b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:09.413680 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50040e05-a8d1-4212-9ccc-a83093d44b7e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:09.413680 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50040e05-a8d1-4212-9ccc-a83093d44b7e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:09.413680 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50040e05-a8d1-4212-9ccc-a83093d44b7e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:09.414379 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50040e05-a8d1-4212-9ccc-a83093d44b7e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:09.414379 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50040e05-a8d1-4212-9ccc-a83093d44b7e None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:09.473511 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b02e30-5526-47be-a3f6-2ff1c59320a9 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:09.477108 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-50040e05-a8d1-4212-9ccc-a83093d44b7e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 21 18:13:09.477108 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:09.477108 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:13:09.477108 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:13:09.477108 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:13:09.477108 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:13:09.477108 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:13:09.477108 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:13:09.477108 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 21 18:13:09.477108 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:09.477108 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:13:09.477108 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:09.477108 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:09.477108 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:09.477108 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:09.477108 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:09.479206 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:09.479206 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:09.479206 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:09.479206 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:09.479206 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:09.479206 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:09.479206 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:09.479206 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:13:09.479206 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:13:09.479206 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:13:09.479206 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:13:09.479206 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:09.479206 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:09.479206 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:09.479206 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:09.479206 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:09.482400 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:09.482400 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:09.482400 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:09.482400 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:09.482400 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:09.482400 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:09.482400 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:09.482400 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:09.482400 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:09.482400 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:09.482400 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:09.482400 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:13:09.482400 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:13:09.482400 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 21 18:13:09.482400 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:09.482400 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 363/707] 10.222.0.87 () {56 vars in 1129 bytes} [Tue Jun 21 18:13:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:13:09.491598 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50040e05-a8d1-4212-9ccc-a83093d44b7e None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:09.588482 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9b89e6-93e6-4e0c-b154-89ca24b845b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:09.589902 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b89e6-93e6-4e0c-b154-89ca24b845b6 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:09.590358 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b89e6-93e6-4e0c-b154-89ca24b845b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:09.590809 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b89e6-93e6-4e0c-b154-89ca24b845b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:09.636048 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9b89e6-93e6-4e0c-b154-89ca24b845b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:09.638752 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9b89e6-93e6-4e0c-b154-89ca24b845b6 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:09.689671 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-ab9b89e6-93e6-4e0c-b154-89ca24b845b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:09.689671 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:09.689671 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:09.689671 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:09.689671 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:09.689671 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:09.689671 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:09.689671 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:09.689671 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:09.689671 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:09.689671 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:09.689671 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:09.689671 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:13:09.689671 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:13:09.689671 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:13:09.690716 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:13:09.690716 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:09.690716 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:09.690716 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:09.690716 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:09.690716 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:09.690716 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:09.690716 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:09.690716 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:09.690716 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:09.690716 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:09.690716 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:09.690716 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:09.690716 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:09.690716 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:09.690716 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:09.690716 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:09.692052 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:13:09.692052 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:13:09.692052 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:09.692052 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:09.696133 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 345/708] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:13:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 471 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:09.699779 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:09.700266 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9b89e6-93e6-4e0c-b154-89ca24b845b6 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:09.823982 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50040e05-a8d1-4212-9ccc-a83093d44b7e None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:09.883962 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:09.914230 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9b89e6-93e6-4e0c-b154-89ca24b845b6 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:09.958918 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c4e6f-2a7a-45f2-9381-0cde2f78e40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:09.959686 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c4e6f-2a7a-45f2-9381-0cde2f78e40b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:09.960012 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c4e6f-2a7a-45f2-9381-0cde2f78e40b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:09.960331 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c4e6f-2a7a-45f2-9381-0cde2f78e40b None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:09.961153 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c4e6f-2a7a-45f2-9381-0cde2f78e40b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:09.961500 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c4e6f-2a7a-45f2-9381-0cde2f78e40b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:09.963162 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c4e6f-2a7a-45f2-9381-0cde2f78e40b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:09.983980 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 364/709] 10.222.0.87 () {56 vars in 1124 bytes} [Tue Jun 21 18:13:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 494 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:09.995219 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c4e6f-2a7a-45f2-9381-0cde2f78e40b None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:09.998345 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d893f6-abc5-4aa4-9cfd-f5cb46db54e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:09.999549 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d893f6-abc5-4aa4-9cfd-f5cb46db54e8 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:10.000037 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d893f6-abc5-4aa4-9cfd-f5cb46db54e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:10.000548 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d893f6-abc5-4aa4-9cfd-f5cb46db54e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:10.002118 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d893f6-abc5-4aa4-9cfd-f5cb46db54e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:10.002824 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d893f6-abc5-4aa4-9cfd-f5cb46db54e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:10.005973 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d893f6-abc5-4aa4-9cfd-f5cb46db54e8 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:10.024605 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 346/710] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:13:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:10.043607 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d893f6-abc5-4aa4-9cfd-f5cb46db54e8 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:10.186165 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d893f6-abc5-4aa4-9cfd-f5cb46db54e8 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:10.249450 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c4e6f-2a7a-45f2-9381-0cde2f78e40b None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:10.269627 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d9029-2851-4617-8e67-d22e4fcfbe8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:10.271243 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d9029-2851-4617-8e67-d22e4fcfbe8a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:10.271771 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d9029-2851-4617-8e67-d22e4fcfbe8a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:10.273156 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d9029-2851-4617-8e67-d22e4fcfbe8a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:10.290117 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5d9029-2851-4617-8e67-d22e4fcfbe8a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:10.293591 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5d9029-2851-4617-8e67-d22e4fcfbe8a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:10.295535 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:10.330832 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-dc5d9029-2851-4617-8e67-d22e4fcfbe8a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:13:10.330832 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:10.330832 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:10.330832 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:10.330832 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:10.330832 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:10.330832 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:10.330832 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:10.330832 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:10.330832 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:10.330832 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:10.330832 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:10.330832 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:13:10.330832 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:13:10.330832 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:13:10.333509 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:13:10.333509 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:10.333509 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:10.333509 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:10.333509 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:10.333509 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:10.333509 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:10.333509 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:10.333509 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:10.333509 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:10.333509 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:10.333509 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:10.333509 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:10.333509 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:10.333509 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:10.333509 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:10.333509 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:10.336045 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:13:10.336045 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:13:10.336045 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:13:10.336045 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:13:10.336045 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:13:10.336045 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:13:10.336045 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:13:10.336045 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:10.336045 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 347/711] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:13:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:13:10.347394 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d9029-2851-4617-8e67-d22e4fcfbe8a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:10.349142 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dbd42c-fdbc-49a2-8689-2a86c41f9541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:10.350555 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dbd42c-fdbc-49a2-8689-2a86c41f9541 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:10.351052 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dbd42c-fdbc-49a2-8689-2a86c41f9541 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:10.351754 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dbd42c-fdbc-49a2-8689-2a86c41f9541 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:10.352816 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dbd42c-fdbc-49a2-8689-2a86c41f9541 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:10.353427 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dbd42c-fdbc-49a2-8689-2a86c41f9541 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:10.355806 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dbd42c-fdbc-49a2-8689-2a86c41f9541 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:10.370912 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 365/712] 10.222.0.87 () {56 vars in 1124 bytes} [Tue Jun 21 18:13:09 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:10.374052 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:10.596317 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d9029-2851-4617-8e67-d22e4fcfbe8a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:10.674289 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8449e28a-cf99-4be7-b8e5-41e54af327d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:10.675442 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449e28a-cf99-4be7-b8e5-41e54af327d6 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:10.675943 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449e28a-cf99-4be7-b8e5-41e54af327d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:10.676426 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449e28a-cf99-4be7-b8e5-41e54af327d6 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:10.677986 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8449e28a-cf99-4be7-b8e5-41e54af327d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:10.678539 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8449e28a-cf99-4be7-b8e5-41e54af327d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:10.681396 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8449e28a-cf99-4be7-b8e5-41e54af327d6 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:10.706150 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 348/713] 10.222.0.87 () {56 vars in 1148 bytes} [Tue Jun 21 18:13:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:10.715861 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dbd42c-fdbc-49a2-8689-2a86c41f9541 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:10.872974 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dbd42c-fdbc-49a2-8689-2a86c41f9541 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:10.913490 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234fa64d-ded2-4dfb-88cd-7518f51a0e41 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:10.914039 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234fa64d-ded2-4dfb-88cd-7518f51a0e41 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:10.914637 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234fa64d-ded2-4dfb-88cd-7518f51a0e41 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:10.917364 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 349/714] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:13:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:13:10.938505 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c05740a-340d-4a1b-aff2-fa3809f14c0f None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:10.939002 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c05740a-340d-4a1b-aff2-fa3809f14c0f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:10.939486 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c05740a-340d-4a1b-aff2-fa3809f14c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:10.945587 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5342f6-56de-4472-b34a-3806804d597b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:10.946832 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5342f6-56de-4472-b34a-3806804d597b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:10.948266 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5342f6-56de-4472-b34a-3806804d597b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:10.948772 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5342f6-56de-4472-b34a-3806804d597b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:10.950495 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5342f6-56de-4472-b34a-3806804d597b None admin] RBAC: Adding query filter params (role.id=98f0943a610f4a9e9f8de54f6e9f717d, scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb, user.id=a1b3fa67905c4027b3656ac36e2b76c9) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:10.951053 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5342f6-56de-4472-b34a-3806804d597b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:10.961525 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5342f6-56de-4472-b34a-3806804d597b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:10.990340 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 366/715] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:13:10 2022] GET /identity/v3/role_assignments?user.id=a1b3fa67905c4027b3656ac36e2b76c9&scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb&role.id=98f0943a610f4a9e9f8de54f6e9f717d => generated 259 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:10.992231 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-0c05740a-340d-4a1b-aff2-fa3809f14c0f None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:13:10.995674 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0c05740a-340d-4a1b-aff2-fa3809f14c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:13:11.169032 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:11.181470 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 350/716] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:13:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:13:11.221912 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5342f6-56de-4472-b34a-3806804d597b None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:11.311549 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:11.401615 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5342f6-56de-4472-b34a-3806804d597b None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:11.455100 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d0501a-7ba0-4ee7-9f37-a1a0fafd53e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:11.455943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d0501a-7ba0-4ee7-9f37-a1a0fafd53e0 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:11.456240 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d0501a-7ba0-4ee7-9f37-a1a0fafd53e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:11.456523 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d0501a-7ba0-4ee7-9f37-a1a0fafd53e0 None admin] PATH_INFO: `/v3/services/network` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:11.457497 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d0501a-7ba0-4ee7-9f37-a1a0fafd53e0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:11.459310 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d0501a-7ba0-4ee7-9f37-a1a0fafd53e0 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:11.479270 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-d9d0501a-7ba0-4ee7-9f37-a1a0fafd53e0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 21 18:13:11.479270 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:11.479270 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:13:11.479270 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:13:11.479270 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:13:11.479270 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:13:11.479270 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:13:11.479270 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:13:11.479270 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 21 18:13:11.479270 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:11.479270 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:13:11.479270 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:11.479270 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:11.479270 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:11.479270 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:11.479270 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:11.479270 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:11.480430 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:11.480430 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:11.480430 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:11.480430 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:11.480430 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:11.480430 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:11.480430 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:13:11.480430 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:13:11.480430 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:13:11.480430 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:13:11.480430 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:11.480430 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:11.480430 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:11.480430 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:11.480430 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:11.480430 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:11.481269 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:11.481269 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:11.481269 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:11.481269 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:11.481269 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:11.481269 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:11.481269 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:11.481269 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:11.481269 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:11.481269 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:11.481269 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:13:11.481269 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:13:11.481269 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 21 18:13:11.481269 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:11.485813 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 367/717] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:13:11 2022] GET /identity/v3/services/network => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:11.491494 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c05740a-340d-4a1b-aff2-fa3809f14c0f None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:11.693653 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c05740a-340d-4a1b-aff2-fa3809f14c0f None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:11.735775 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da0db5c-14e8-4663-9ee0-202678c4b4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:11.736862 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0db5c-14e8-4663-9ee0-202678c4b4c7 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:11.737391 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0db5c-14e8-4663-9ee0-202678c4b4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:11.737900 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0db5c-14e8-4663-9ee0-202678c4b4c7 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:11.742675 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da0db5c-14e8-4663-9ee0-202678c4b4c7 None admin] RBAC: Adding query filter params (name=network) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:11.743338 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da0db5c-14e8-4663-9ee0-202678c4b4c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:11.745040 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da0db5c-14e8-4663-9ee0-202678c4b4c7 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:11.755478 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 351/718] 10.222.0.87 () {56 vars in 1112 bytes} [Tue Jun 21 18:13:11 2022] GET /identity/v3/services?name=network => generated 124 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:11.766845 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d0501a-7ba0-4ee7-9f37-a1a0fafd53e0 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:11.945342 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d0501a-7ba0-4ee7-9f37-a1a0fafd53e0 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:11.963757 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:12.002899 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53740998-e035-4e01-b896-f1533d6a1202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:12.003508 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53740998-e035-4e01-b896-f1533d6a1202 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:12.003795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53740998-e035-4e01-b896-f1533d6a1202 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:12.004075 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53740998-e035-4e01-b896-f1533d6a1202 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:12.004826 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53740998-e035-4e01-b896-f1533d6a1202 None admin] RBAC: Adding query filter params (type=network) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:12.005140 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53740998-e035-4e01-b896-f1533d6a1202 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:12.006623 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53740998-e035-4e01-b896-f1533d6a1202 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:12.018327 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 368/719] 10.222.0.87 () {56 vars in 1112 bytes} [Tue Jun 21 18:13:11 2022] GET /identity/v3/services?type=network => generated 124 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:12.264522 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960babcd-82a7-4cb6-ab90-b7cfd137eebd None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:12.264710 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960babcd-82a7-4cb6-ab90-b7cfd137eebd None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:12.265086 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960babcd-82a7-4cb6-ab90-b7cfd137eebd None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:12.268098 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 352/720] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:13:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:13:12.273928 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d52c8c-c738-496b-a3ea-1d4633c6e569 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:12.274591 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d52c8c-c738-496b-a3ea-1d4633c6e569 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:12.275269 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d52c8c-c738-496b-a3ea-1d4633c6e569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:12.300758 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-42d52c8c-c738-496b-a3ea-1d4633c6e569 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:13:12.302442 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-42d52c8c-c738-496b-a3ea-1d4633c6e569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:13:12.413705 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 369/721] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:13:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:13:12.439261 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960babcd-82a7-4cb6-ab90-b7cfd137eebd None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:12.549986 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba019a0b-ad66-4744-a616-467bdc5dda56 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:12.550389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba019a0b-ad66-4744-a616-467bdc5dda56 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:12.550774 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba019a0b-ad66-4744-a616-467bdc5dda56 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:12.552744 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 370/722] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:13:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:13:12.554920 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:12.564026 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc922c-fe3c-4269-8208-e351ed50e07d None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:12.564385 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc922c-fe3c-4269-8208-e351ed50e07d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:12.564602 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc922c-fe3c-4269-8208-e351ed50e07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:12.599105 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-cbfc922c-fe3c-4269-8208-e351ed50e07d None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:13:12.600999 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-cbfc922c-fe3c-4269-8208-e351ed50e07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:13:12.621441 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960babcd-82a7-4cb6-ab90-b7cfd137eebd None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:12.690127 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68d2b9-3757-4e1b-b350-8fc1199944fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:12.691584 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68d2b9-3757-4e1b-b350-8fc1199944fa None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:12.692075 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68d2b9-3757-4e1b-b350-8fc1199944fa None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:12.692566 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68d2b9-3757-4e1b-b350-8fc1199944fa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:12.718578 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d68d2b9-3757-4e1b-b350-8fc1199944fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:13:12.747188 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-7d68d2b9-3757-4e1b-b350-8fc1199944fa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:13:12.747188 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:12.747188 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:12.747188 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:12.747188 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:12.747188 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:12.747188 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:12.747188 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:12.747188 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:12.747188 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:12.747188 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:12.747188 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:12.747188 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:13:12.747188 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:13:12.747188 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:13:12.747188 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:13:12.748201 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:13:12.748201 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:13:12.748201 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:13:12.748201 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:13:12.748201 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:12.748201 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:12.748201 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:12.748201 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:12.748201 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:12.748201 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:12.748201 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:12.748201 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:12.748201 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:12.748201 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:12.748201 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:12.748201 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:12.749288 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:12.749288 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:12.749288 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:12.749288 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:12.749288 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:13:12.749288 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:13:12.749288 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:13:12.749288 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:13:12.749288 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:13:12.749288 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:13:12.749288 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:13:12.749288 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:12.750023 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 353/723] 10.222.0.87 () {56 vars in 1102 bytes} [Tue Jun 21 18:13:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:12.756520 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68d2b9-3757-4e1b-b350-8fc1199944fa None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:12.847286 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 371/724] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:13:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:13:12.897630 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc922c-fe3c-4269-8208-e351ed50e07d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:12.981334 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:13.037958 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc922c-fe3c-4269-8208-e351ed50e07d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:13.092630 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69350568-fd8c-4c89-bd75-006e26ae205d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:13.094506 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69350568-fd8c-4c89-bd75-006e26ae205d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:13.095232 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69350568-fd8c-4c89-bd75-006e26ae205d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:13.095775 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69350568-fd8c-4c89-bd75-006e26ae205d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:13.114275 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69350568-fd8c-4c89-bd75-006e26ae205d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:13:13.125463 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-69350568-fd8c-4c89-bd75-006e26ae205d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:13:13.125463 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:13.125463 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:13.125463 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:13.125463 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:13.125463 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:13.125463 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:13.125463 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:13.125463 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:13.125463 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:13.125463 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:13.125463 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:13.125463 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:13:13.125463 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:13:13.125463 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:13:13.125463 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:13:13.126946 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:13:13.126946 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:13:13.126946 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:13:13.126946 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:13:13.126946 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:13.126946 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:13.126946 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:13.126946 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:13.126946 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:13.126946 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:13.126946 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:13.126946 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:13.126946 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:13.126946 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:13.126946 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:13.126946 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:13.127971 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:13.127971 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:13.127971 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:13.127971 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:13.127971 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:13:13.127971 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:13:13.127971 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:13:13.127971 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:13:13.127971 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:13:13.127971 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:13:13.127971 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:13:13.127971 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:13.136103 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 372/725] 10.222.0.87 () {56 vars in 1096 bytes} [Tue Jun 21 18:13:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:13:13.137922 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69350568-fd8c-4c89-bd75-006e26ae205d None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:13.165307 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68d2b9-3757-4e1b-b350-8fc1199944fa None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:13.269137 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:13.294281 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:13.312182 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69350568-fd8c-4c89-bd75-006e26ae205d None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:13.314893 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace3fa4-941c-4e98-8ad3-211daa55a667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:13.316011 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace3fa4-941c-4e98-8ad3-211daa55a667 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:13.316488 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace3fa4-941c-4e98-8ad3-211daa55a667 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:13.316971 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace3fa4-941c-4e98-8ad3-211daa55a667 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:13.318450 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ace3fa4-941c-4e98-8ad3-211daa55a667 None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:13.319008 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ace3fa4-941c-4e98-8ad3-211daa55a667 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:13.321760 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ace3fa4-941c-4e98-8ad3-211daa55a667 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:13.342377 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 354/726] 10.222.0.87 () {56 vars in 1103 bytes} [Tue Jun 21 18:13:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 587 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:13:13.354828 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace3fa4-941c-4e98-8ad3-211daa55a667 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:13.455501 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b1c97-950f-415c-bc59-4f6575b198fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:13.457213 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b1c97-950f-415c-bc59-4f6575b198fd None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:13.457955 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b1c97-950f-415c-bc59-4f6575b198fd None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:13.458653 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b1c97-950f-415c-bc59-4f6575b198fd None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:13.460296 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133b1c97-950f-415c-bc59-4f6575b198fd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:13.461090 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133b1c97-950f-415c-bc59-4f6575b198fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:13.464765 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133b1c97-950f-415c-bc59-4f6575b198fd None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:13.483875 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 373/727] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:13:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:13.491459 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b1c97-950f-415c-bc59-4f6575b198fd None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:13.733797 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace3fa4-941c-4e98-8ad3-211daa55a667 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:13.794957 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b1c97-950f-415c-bc59-4f6575b198fd None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:13.821455 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5461d480-b359-4522-8e04-8dacb73d01a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:13.823015 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461d480-b359-4522-8e04-8dacb73d01a9 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:13.823841 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461d480-b359-4522-8e04-8dacb73d01a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:13.824607 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461d480-b359-4522-8e04-8dacb73d01a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:13.855497 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5461d480-b359-4522-8e04-8dacb73d01a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:13.859132 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5461d480-b359-4522-8e04-8dacb73d01a9 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:13.861621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3514399d-02fe-4071-944f-630c1b801846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:13.862557 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514399d-02fe-4071-944f-630c1b801846 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:13.862906 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514399d-02fe-4071-944f-630c1b801846 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:13.863269 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514399d-02fe-4071-944f-630c1b801846 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:13.881572 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3514399d-02fe-4071-944f-630c1b801846 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:13.884445 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3514399d-02fe-4071-944f-630c1b801846 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:13.897828 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-5461d480-b359-4522-8e04-8dacb73d01a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:13.897828 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:13.897828 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:13.897828 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:13.897828 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:13.897828 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:13.897828 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:13.897828 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:13.897828 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:13.897828 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:13.897828 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:13.897828 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:13.897828 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:13:13.897828 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:13:13.897828 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:13:13.899719 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:13:13.899719 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:13.899719 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:13.899719 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:13.899719 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:13.899719 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:13.899719 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:13.899719 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:13.899719 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:13.899719 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:13.899719 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:13.899719 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:13.899719 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:13.899719 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:13.899719 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:13.899719 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:13.899719 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:13.901495 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:13:13.901495 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:13:13.901495 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:13.901495 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:13.905827 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 355/728] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:13:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 554 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:13:13.908647 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0de495-4ecf-403e-a72a-0f11f39f982f None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:13.909468 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0de495-4ecf-403e-a72a-0f11f39f982f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:13.910268 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0de495-4ecf-403e-a72a-0f11f39f982f None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:13.911528 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-3514399d-02fe-4071-944f-630c1b801846 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:13.911528 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:13.911528 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:13.911528 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:13.911528 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:13.911528 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:13.911528 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:13.911528 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:13.911528 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:13.911528 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:13.911528 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:13.911528 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:13.911528 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:13:13.911528 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:13:13.911528 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:13:13.913134 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:13:13.913134 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:13.913134 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:13.913134 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:13.913134 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:13.913134 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:13.913134 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:13.913134 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:13.913134 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:13.913134 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:13.913134 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:13.913134 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:13.913134 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:13.913134 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:13.913134 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:13.913134 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:13.913134 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:13.914875 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:13:13.914875 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:13:13.914875 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:13.914875 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:13.917797 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 374/729] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:13:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 423 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:13.918834 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3514399d-02fe-4071-944f-630c1b801846 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:13.919038 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 356/730] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:13:13 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:13:13.920625 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0de495-4ecf-403e-a72a-0f11f39f982f None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:13.937681 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:13.992776 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:14.068333 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3514399d-02fe-4071-944f-630c1b801846 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:14.114563 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f02cddf-bc65-4b52-b10d-cca28be85e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:14.115274 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02cddf-bc65-4b52-b10d-cca28be85e44 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:14.115561 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02cddf-bc65-4b52-b10d-cca28be85e44 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:14.115845 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02cddf-bc65-4b52-b10d-cca28be85e44 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:14.116616 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f02cddf-bc65-4b52-b10d-cca28be85e44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:14.116929 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f02cddf-bc65-4b52-b10d-cca28be85e44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:14.118661 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f02cddf-bc65-4b52-b10d-cca28be85e44 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:14.126270 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0de495-4ecf-403e-a72a-0f11f39f982f None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:14.139159 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 375/731] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:13:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:14.141965 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f80a2a7-42f1-4ff6-9913-7af55d8f2e63 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:14.142460 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f80a2a7-42f1-4ff6-9913-7af55d8f2e63 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:14.142936 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f80a2a7-42f1-4ff6-9913-7af55d8f2e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:14.165970 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-7f80a2a7-42f1-4ff6-9913-7af55d8f2e63 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:13:14.167342 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7f80a2a7-42f1-4ff6-9913-7af55d8f2e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:13:14.182717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcf400f-87ed-4811-8fc1-149bf1beb0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:14.184348 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf400f-87ed-4811-8fc1-149bf1beb0db None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:14.185326 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf400f-87ed-4811-8fc1-149bf1beb0db None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:14.186175 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf400f-87ed-4811-8fc1-149bf1beb0db None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:14.187708 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcf400f-87ed-4811-8fc1-149bf1beb0db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:14.188444 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcf400f-87ed-4811-8fc1-149bf1beb0db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:14.191239 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcf400f-87ed-4811-8fc1-149bf1beb0db None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:14.221694 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 357/732] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:13:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:14.221694 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d94349-79e1-4107-9728-7ee635d49271 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:14.221694 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d94349-79e1-4107-9728-7ee635d49271 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:14.221694 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d94349-79e1-4107-9728-7ee635d49271 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:14.222919 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 358/733] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:13:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:13:14.225825 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d94349-79e1-4107-9728-7ee635d49271 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:14.260787 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:14.273706 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 376/734] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:13:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:13:14.275318 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f80a2a7-42f1-4ff6-9913-7af55d8f2e63 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:14.402349 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:14.420279 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d94349-79e1-4107-9728-7ee635d49271 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:14.456571 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f80a2a7-42f1-4ff6-9913-7af55d8f2e63 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:14.476986 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc4b595-be04-4734-b587-6ff572edc286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:14.478459 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4b595-be04-4734-b587-6ff572edc286 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:14.478743 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4b595-be04-4734-b587-6ff572edc286 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:14.479142 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4b595-be04-4734-b587-6ff572edc286 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:14.489746 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc4b595-be04-4734-b587-6ff572edc286 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:14.491663 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc4b595-be04-4734-b587-6ff572edc286 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:14.513737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-cfc4b595-be04-4734-b587-6ff572edc286 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 18:13:14.513737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:14.513737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:14.513737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:14.513737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:14.513737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:14.513737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:14.513737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:14.513737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:14.513737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:14.513737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:14.513737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:14.513737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:13:14.513737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:13:14.513737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:13:14.513737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:13:14.514737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:14.514737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:14.514737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:13:14.514737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:13:14.514737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:13:14.514737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:13:14.514737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:14.514737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:14.514737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:14.514737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:14.514737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:14.514737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:14.514737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:14.514737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:14.514737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:14.514737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:14.514737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:14.514737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:14.515699 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:14.515699 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:14.515699 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:13:14.515699 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:13:14.515699 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:13:14.515699 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:13:14.515699 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:13:14.515699 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:13:14.515699 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 18:13:14.515699 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:14.516315 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 359/735] 10.222.0.87 () {56 vars in 1117 bytes} [Tue Jun 21 18:13:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:14.518518 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eccde8-d01b-4287-bbc2-ec6b475a2724 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:14.519015 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eccde8-d01b-4287-bbc2-ec6b475a2724 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:14.519511 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eccde8-d01b-4287-bbc2-ec6b475a2724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:14.532422 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fefdd26-c198-40d0-9cfa-261a502b1cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:14.533771 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fefdd26-c198-40d0-9cfa-261a502b1cd2 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:14.534223 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fefdd26-c198-40d0-9cfa-261a502b1cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:14.534714 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fefdd26-c198-40d0-9cfa-261a502b1cd2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:14.550024 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fefdd26-c198-40d0-9cfa-261a502b1cd2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:14.553023 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f0eccde8-d01b-4287-bbc2-ec6b475a2724 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:13:14.553181 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fefdd26-c198-40d0-9cfa-261a502b1cd2 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:14.555064 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f0eccde8-d01b-4287-bbc2-ec6b475a2724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:13:14.578955 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-3fefdd26-c198-40d0-9cfa-261a502b1cd2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 21 18:13:14.578955 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:14.578955 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:14.578955 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:14.578955 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:14.578955 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:14.578955 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:14.578955 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:14.578955 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:14.578955 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:14.578955 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:14.578955 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:14.578955 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:13:14.578955 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:13:14.578955 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:13:14.578955 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:13:14.580108 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:14.580108 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:14.580108 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:13:14.580108 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:13:14.580108 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:13:14.580108 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:13:14.580108 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:14.580108 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:14.580108 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:14.580108 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:14.580108 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:14.580108 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:14.580108 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:14.580108 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:14.580108 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:14.580108 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:14.580108 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:14.580108 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:14.581142 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:14.581142 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:14.581142 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:13:14.581142 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:13:14.581142 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:13:14.581142 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:13:14.581142 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:13:14.581142 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:13:14.581142 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 21 18:13:14.581142 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:14.588440 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 377/736] 10.222.0.87 () {56 vars in 1096 bytes} [Tue Jun 21 18:13:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:14.588440 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fefdd26-c198-40d0-9cfa-261a502b1cd2 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:14.681690 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 360/737] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:13:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:13:14.681690 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eccde8-d01b-4287-bbc2-ec6b475a2724 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:14.715595 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:14.805297 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fefdd26-c198-40d0-9cfa-261a502b1cd2 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:14.878263 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65ae2b0-330a-43d7-8d10-470bfe383c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:14.879854 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ae2b0-330a-43d7-8d10-470bfe383c04 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:14.880437 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ae2b0-330a-43d7-8d10-470bfe383c04 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:14.881009 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ae2b0-330a-43d7-8d10-470bfe383c04 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:14.887487 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:14.906308 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65ae2b0-330a-43d7-8d10-470bfe383c04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:13:14.909156 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eccde8-d01b-4287-bbc2-ec6b475a2724 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:14.923236 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-f65ae2b0-330a-43d7-8d10-470bfe383c04 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:13:14.923236 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:14.923236 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:14.923236 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:14.923236 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:14.923236 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:14.923236 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:14.923236 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:14.923236 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:14.923236 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:14.923236 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:14.923236 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:14.923236 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:13:14.923236 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:13:14.923236 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:13:14.923236 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:13:14.925146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:13:14.925146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:13:14.925146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:13:14.925146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:13:14.925146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:14.925146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:14.925146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:14.925146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:14.925146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:14.925146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:14.925146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:14.925146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:14.925146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:14.925146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:14.925146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:14.925146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:14.926879 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:14.926879 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:14.926879 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:14.926879 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:14.926879 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:13:14.926879 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:13:14.926879 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:13:14.926879 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:13:14.926879 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:13:14.926879 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:13:14.926879 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:13:14.926879 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:14.926879 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 378/738] 10.222.0.87 () {56 vars in 1096 bytes} [Tue Jun 21 18:13:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:14.928262 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65ae2b0-330a-43d7-8d10-470bfe383c04 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:14.980232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ae6825-04f2-4a44-ae6c-5c42539bde1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:14.981408 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae6825-04f2-4a44-ae6c-5c42539bde1c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:14.981983 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae6825-04f2-4a44-ae6c-5c42539bde1c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:14.982479 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae6825-04f2-4a44-ae6c-5c42539bde1c None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:14.984432 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ae6825-04f2-4a44-ae6c-5c42539bde1c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:14.984978 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ae6825-04f2-4a44-ae6c-5c42539bde1c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:14.987857 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ae6825-04f2-4a44-ae6c-5c42539bde1c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:15.024368 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 361/739] 10.222.0.87 () {56 vars in 1149 bytes} [Tue Jun 21 18:13:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:15.027172 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ae6825-04f2-4a44-ae6c-5c42539bde1c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:15.151676 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65ae2b0-330a-43d7-8d10-470bfe383c04 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:15.192649 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:15.229002 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e8f5ca-c1e6-4b44-9049-d9bba66cc20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:15.230238 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8f5ca-c1e6-4b44-9049-d9bba66cc20c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:15.230726 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8f5ca-c1e6-4b44-9049-d9bba66cc20c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:15.231505 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8f5ca-c1e6-4b44-9049-d9bba66cc20c None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:15.233875 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e8f5ca-c1e6-4b44-9049-d9bba66cc20c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:15.234561 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e8f5ca-c1e6-4b44-9049-d9bba66cc20c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:15.237173 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e8f5ca-c1e6-4b44-9049-d9bba66cc20c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:15.251736 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ae6825-04f2-4a44-ae6c-5c42539bde1c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:15.277724 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 379/740] 10.222.0.87 () {56 vars in 1135 bytes} [Tue Jun 21 18:13:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:15.277724 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e8f5ca-c1e6-4b44-9049-d9bba66cc20c None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:15.329525 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fcb5c7-ac7c-4b07-bd70-921835b953aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:15.331143 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fcb5c7-ac7c-4b07-bd70-921835b953aa None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:15.331902 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fcb5c7-ac7c-4b07-bd70-921835b953aa None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:15.332694 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fcb5c7-ac7c-4b07-bd70-921835b953aa None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:15.334069 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fcb5c7-ac7c-4b07-bd70-921835b953aa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:15.336459 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fcb5c7-ac7c-4b07-bd70-921835b953aa None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:15.369308 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 362/741] 10.222.0.87 () {60 vars in 1141 bytes} [Tue Jun 21 18:13:15 2022] POST /identity/v3/services => generated 251 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:13:15.371885 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fcb5c7-ac7c-4b07-bd70-921835b953aa None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:15.385867 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:15.562721 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e8f5ca-c1e6-4b44-9049-d9bba66cc20c None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:15.635868 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83960a7e-38b3-4e96-8ad3-552eb664ca97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:15.636980 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83960a7e-38b3-4e96-8ad3-552eb664ca97 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:15.637277 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83960a7e-38b3-4e96-8ad3-552eb664ca97 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:15.637667 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83960a7e-38b3-4e96-8ad3-552eb664ca97 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:15.638884 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83960a7e-38b3-4e96-8ad3-552eb664ca97 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:15.639382 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83960a7e-38b3-4e96-8ad3-552eb664ca97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:15.642186 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83960a7e-38b3-4e96-8ad3-552eb664ca97 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:15.660644 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 380/742] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:13:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:15.662130 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83960a7e-38b3-4e96-8ad3-552eb664ca97 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:15.743840 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:15.760905 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fcb5c7-ac7c-4b07-bd70-921835b953aa None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:15.866604 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1de8925-72f2-488f-b1bd-89ca8e60812e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:15.868578 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de8925-72f2-488f-b1bd-89ca8e60812e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:15.869472 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de8925-72f2-488f-b1bd-89ca8e60812e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:15.870328 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de8925-72f2-488f-b1bd-89ca8e60812e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:15.897754 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83960a7e-38b3-4e96-8ad3-552eb664ca97 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:15.976029 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4657c682-07a5-4611-8607-ba75e93342e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:15.977527 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4657c682-07a5-4611-8607-ba75e93342e8 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:15.978286 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4657c682-07a5-4611-8607-ba75e93342e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:15.978984 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4657c682-07a5-4611-8607-ba75e93342e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:16.030857 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1de8925-72f2-488f-b1bd-89ca8e60812e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:16.032501 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1de8925-72f2-488f-b1bd-89ca8e60812e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:16.037392 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:16.121823 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4657c682-07a5-4611-8607-ba75e93342e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:16.123501 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4657c682-07a5-4611-8607-ba75e93342e8 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:16.180563 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:16.235453 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 363/743] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:13:15 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 860 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:13:16.235453 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1dcca-db5a-454b-8c96-283548a54c79 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:16.235453 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1dcca-db5a-454b-8c96-283548a54c79 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:16.235453 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1dcca-db5a-454b-8c96-283548a54c79 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:16.236459 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 364/744] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:13:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:13:16.238294 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c1dcca-db5a-454b-8c96-283548a54c79 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:16.339970 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 381/745] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:13:15 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 679 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:13:16.342727 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b480b5e-9bef-41a3-969a-9374c1a4321e None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:16.343511 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b480b5e-9bef-41a3-969a-9374c1a4321e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:16.344051 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b480b5e-9bef-41a3-969a-9374c1a4321e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:16.369283 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-7b480b5e-9bef-41a3-969a-9374c1a4321e None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:13:16.371683 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7b480b5e-9bef-41a3-969a-9374c1a4321e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:13:16.436423 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c1dcca-db5a-454b-8c96-283548a54c79 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:16.441278 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:16.513225 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 382/746] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:13:16 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:13:16.513945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a486ea7-cd62-41b5-82b6-e3d887e1f3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:16.515360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b480b5e-9bef-41a3-969a-9374c1a4321e None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:16.515920 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a486ea7-cd62-41b5-82b6-e3d887e1f3d0 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:16.516637 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a486ea7-cd62-41b5-82b6-e3d887e1f3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:16.517371 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a486ea7-cd62-41b5-82b6-e3d887e1f3d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:16.537508 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a486ea7-cd62-41b5-82b6-e3d887e1f3d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:16.539444 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a486ea7-cd62-41b5-82b6-e3d887e1f3d0 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:16.572619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-6a486ea7-cd62-41b5-82b6-e3d887e1f3d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:16.572619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:16.572619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:16.572619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:16.572619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:16.572619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:16.572619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:16.572619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:16.572619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:16.572619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:16.572619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:16.572619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:16.572619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:13:16.572619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:13:16.572619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:13:16.581009 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:13:16.581009 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:16.581009 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:16.581009 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:16.581009 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:16.581009 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:16.581009 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:16.581009 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:16.581009 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:16.581009 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:16.581009 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:16.581009 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:16.581009 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:16.581009 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:16.581009 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:16.581009 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:16.581009 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:16.583764 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:13:16.583764 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:13:16.583764 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:16.583764 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:16.583764 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 365/747] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:13:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:16.583764 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a486ea7-cd62-41b5-82b6-e3d887e1f3d0 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:16.663541 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:16.726869 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b480b5e-9bef-41a3-969a-9374c1a4321e None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:16.794289 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10d4536-180e-46eb-924a-c198d22a44a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:16.795722 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d4536-180e-46eb-924a-c198d22a44a0 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:16.796194 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d4536-180e-46eb-924a-c198d22a44a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:16.796679 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d4536-180e-46eb-924a-c198d22a44a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:16.802915 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a486ea7-cd62-41b5-82b6-e3d887e1f3d0 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:16.817897 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10d4536-180e-46eb-924a-c198d22a44a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:16.820723 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10d4536-180e-46eb-924a-c198d22a44a0 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:16.859848 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-e10d4536-180e-46eb-924a-c198d22a44a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:16.859848 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:16.859848 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:16.859848 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:16.859848 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:16.859848 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:16.859848 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:16.859848 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:16.859848 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:16.859848 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:16.859848 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:16.859848 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:16.859848 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:13:16.859848 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:13:16.859848 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:13:16.861629 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:13:16.861629 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:16.861629 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:16.861629 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:16.861629 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:16.861629 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:16.861629 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:16.861629 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:16.861629 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:16.861629 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:16.861629 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:16.861629 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:16.861629 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:16.861629 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:16.861629 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:16.861629 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:16.861629 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:16.865128 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:13:16.865128 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:13:16.865128 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:16.865128 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:16.865128 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 383/748] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:13:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:16.865128 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10d4536-180e-46eb-924a-c198d22a44a0 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:16.886001 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7944b6a8-d3b1-4653-a717-41d597ee0eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:16.887541 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7944b6a8-d3b1-4653-a717-41d597ee0eb3 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:16.887725 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7944b6a8-d3b1-4653-a717-41d597ee0eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:16.889947 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7944b6a8-d3b1-4653-a717-41d597ee0eb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:16.917179 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7944b6a8-d3b1-4653-a717-41d597ee0eb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:16.920103 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7944b6a8-d3b1-4653-a717-41d597ee0eb3 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:16.951170 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-7944b6a8-d3b1-4653-a717-41d597ee0eb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:16.951170 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:16.951170 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:16.951170 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:16.951170 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:16.951170 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:16.951170 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:16.951170 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:16.951170 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:16.951170 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:16.951170 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:16.951170 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:16.951170 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:13:16.951170 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:13:16.951170 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:13:16.954304 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:13:16.954304 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:16.954304 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:16.954304 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:16.954304 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:16.954304 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:16.954304 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:16.954304 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:16.954304 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:16.954304 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:16.954304 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:16.954304 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:16.954304 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:16.954304 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:16.954304 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:16.954304 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:16.954304 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:16.956065 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:13:16.956065 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:13:16.956065 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:16.956065 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:16.956065 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 366/749] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:13:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:16.956065 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7944b6a8-d3b1-4653-a717-41d597ee0eb3 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:16.982909 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:17.067380 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10d4536-180e-46eb-924a-c198d22a44a0 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:17.095065 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:17.132660 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f778f33-553f-4ee1-bbc8-c235292f3568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:17.133368 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f778f33-553f-4ee1-bbc8-c235292f3568 None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:17.133738 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f778f33-553f-4ee1-bbc8-c235292f3568 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:17.134048 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f778f33-553f-4ee1-bbc8-c235292f3568 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:17.134922 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f778f33-553f-4ee1-bbc8-c235292f3568 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:17.136669 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f778f33-553f-4ee1-bbc8-c235292f3568 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:17.160275 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7944b6a8-d3b1-4653-a717-41d597ee0eb3 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:17.162314 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 384/750] 10.222.0.87 () {60 vars in 1141 bytes} [Tue Jun 21 18:13:16 2022] POST /identity/v3/services => generated 240 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:13:17.171200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f778f33-553f-4ee1-bbc8-c235292f3568 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:17.264914 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679b32b8-0aaa-4186-bde5-e6307fab2dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:17.266805 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b32b8-0aaa-4186-bde5-e6307fab2dfb None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:17.267734 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b32b8-0aaa-4186-bde5-e6307fab2dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:17.268593 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b32b8-0aaa-4186-bde5-e6307fab2dfb None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:17.270448 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679b32b8-0aaa-4186-bde5-e6307fab2dfb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:17.271425 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679b32b8-0aaa-4186-bde5-e6307fab2dfb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:17.275929 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679b32b8-0aaa-4186-bde5-e6307fab2dfb None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:17.309582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 367/751] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:13:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:17.311531 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679b32b8-0aaa-4186-bde5-e6307fab2dfb None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:17.433726 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f778f33-553f-4ee1-bbc8-c235292f3568 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:17.460605 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:17.521781 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62751460-7b4e-4038-9c83-5a97372a00fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:17.523193 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62751460-7b4e-4038-9c83-5a97372a00fa None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:17.523669 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62751460-7b4e-4038-9c83-5a97372a00fa None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:17.524323 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62751460-7b4e-4038-9c83-5a97372a00fa None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:17.526092 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62751460-7b4e-4038-9c83-5a97372a00fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:17.527558 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62751460-7b4e-4038-9c83-5a97372a00fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:17.530606 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62751460-7b4e-4038-9c83-5a97372a00fa None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:17.531076 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679b32b8-0aaa-4186-bde5-e6307fab2dfb None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:17.561845 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 385/752] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:13:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:17.564444 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62751460-7b4e-4038-9c83-5a97372a00fa None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:17.610391 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e6937a-faf7-493d-92a4-2e081dfdc24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:17.611722 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6937a-faf7-493d-92a4-2e081dfdc24e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:17.612303 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6937a-faf7-493d-92a4-2e081dfdc24e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:17.612951 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6937a-faf7-493d-92a4-2e081dfdc24e None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:17.614638 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e6937a-faf7-493d-92a4-2e081dfdc24e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:17.615279 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e6937a-faf7-493d-92a4-2e081dfdc24e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:17.618211 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e6937a-faf7-493d-92a4-2e081dfdc24e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:17.645108 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:17.650265 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 368/753] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:13:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:17.652815 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e6937a-faf7-493d-92a4-2e081dfdc24e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:17.792400 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62751460-7b4e-4038-9c83-5a97372a00fa None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:17.875148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e6937a-faf7-493d-92a4-2e081dfdc24e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:17.889384 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf84a17b-0a7e-4cc8-8013-fd27d543503c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:17.892656 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84a17b-0a7e-4cc8-8013-fd27d543503c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:17.893435 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84a17b-0a7e-4cc8-8013-fd27d543503c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:17.893981 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84a17b-0a7e-4cc8-8013-fd27d543503c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:17.912411 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf84a17b-0a7e-4cc8-8013-fd27d543503c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:17.916135 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf84a17b-0a7e-4cc8-8013-fd27d543503c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:17.929094 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:17.950605 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-bf84a17b-0a7e-4cc8-8013-fd27d543503c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:13:17.950605 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:17.950605 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:17.950605 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:17.950605 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:17.950605 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:17.950605 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:17.950605 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:17.950605 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:17.950605 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:17.950605 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:17.950605 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:17.950605 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:13:17.950605 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:13:17.950605 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:13:17.950605 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:13:17.952723 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:17.952723 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:17.952723 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:13:17.952723 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:13:17.952723 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:13:17.952723 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:13:17.952723 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:17.952723 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:17.952723 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:17.952723 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:17.952723 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:17.952723 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:17.952723 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:17.952723 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:17.952723 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:17.952723 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:17.952723 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:17.952723 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:17.954734 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:17.954734 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:17.954734 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:13:17.954734 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:13:17.954734 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:13:17.954734 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:13:17.954734 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:13:17.954734 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:13:17.954734 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:13:17.954734 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:17.955814 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a16e133-d9d8-4921-8f98-e79f3f99d160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:17.955814 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16e133-d9d8-4921-8f98-e79f3f99d160 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:17.955814 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16e133-d9d8-4921-8f98-e79f3f99d160 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:17.955814 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16e133-d9d8-4921-8f98-e79f3f99d160 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:17.965816 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 386/754] 10.222.0.87 () {56 vars in 1093 bytes} [Tue Jun 21 18:13:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 398 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:17.968123 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf84a17b-0a7e-4cc8-8013-fd27d543503c None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:17.970964 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a16e133-d9d8-4921-8f98-e79f3f99d160 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:17.974510 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a16e133-d9d8-4921-8f98-e79f3f99d160 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:18.006487 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-3a16e133-d9d8-4921-8f98-e79f3f99d160 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:13:18.006487 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:18.006487 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:18.006487 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:18.006487 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:18.006487 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:18.006487 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:18.006487 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:18.006487 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:18.006487 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:18.006487 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:18.006487 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:18.006487 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:13:18.006487 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:13:18.006487 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:13:18.008497 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:13:18.008497 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:18.008497 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:18.008497 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:18.008497 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:18.008497 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:18.008497 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:18.008497 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:18.008497 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:18.008497 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:18.008497 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:18.008497 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:18.008497 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:18.008497 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:18.008497 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:18.008497 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:18.008497 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:18.011247 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:13:18.011247 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:13:18.011247 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:13:18.011247 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:13:18.011247 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:13:18.011247 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:13:18.011247 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:13:18.011247 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:18.011247 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 369/755] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:13:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:18.012879 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a16e133-d9d8-4921-8f98-e79f3f99d160 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:18.132913 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf84a17b-0a7e-4cc8-8013-fd27d543503c None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:18.148516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:18.189998 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af61f1ac-054e-45a6-9124-eab3c66649f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:18.192655 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af61f1ac-054e-45a6-9124-eab3c66649f3 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:18.193591 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af61f1ac-054e-45a6-9124-eab3c66649f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:18.195827 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a16e133-d9d8-4921-8f98-e79f3f99d160 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:18.198338 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af61f1ac-054e-45a6-9124-eab3c66649f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:18.222534 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af61f1ac-054e-45a6-9124-eab3c66649f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:18.226409 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af61f1ac-054e-45a6-9124-eab3c66649f3 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:18.257514 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-af61f1ac-054e-45a6-9124-eab3c66649f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:13:18.257514 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:18.257514 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:18.257514 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:18.257514 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:18.257514 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:18.257514 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:18.257514 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:18.257514 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:18.257514 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:18.257514 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:18.257514 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:18.257514 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:13:18.257514 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:13:18.257514 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:13:18.260265 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:13:18.260265 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:18.260265 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:18.260265 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:18.260265 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:18.260265 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:18.260265 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:18.260265 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:18.260265 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:18.260265 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:18.260265 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:18.260265 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:18.260265 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:18.260265 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:18.260265 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:18.260265 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:18.260265 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:18.262638 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:13:18.262638 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:13:18.262638 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:13:18.262638 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:13:18.262638 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:13:18.262638 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:13:18.262638 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:13:18.262638 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:18.266653 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450dd964-4331-4465-96d9-5ede7515bd06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:18.267737 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 387/756] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:13:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:18.269701 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af61f1ac-054e-45a6-9124-eab3c66649f3 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:18.275558 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450dd964-4331-4465-96d9-5ede7515bd06 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:18.275994 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450dd964-4331-4465-96d9-5ede7515bd06 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:18.276430 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450dd964-4331-4465-96d9-5ede7515bd06 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:18.278144 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450dd964-4331-4465-96d9-5ede7515bd06 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:18.278679 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450dd964-4331-4465-96d9-5ede7515bd06 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:18.282192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450dd964-4331-4465-96d9-5ede7515bd06 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:18.317121 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 370/757] 10.222.0.87 () {56 vars in 1133 bytes} [Tue Jun 21 18:13:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:18.318784 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450dd964-4331-4465-96d9-5ede7515bd06 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:18.406510 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:18.498305 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450dd964-4331-4465-96d9-5ede7515bd06 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:18.503904 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af61f1ac-054e-45a6-9124-eab3c66649f3 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:18.563162 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23951a25-fe3f-4675-817f-a5941c08d4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:18.564384 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23951a25-fe3f-4675-817f-a5941c08d4cd None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:18.564952 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23951a25-fe3f-4675-817f-a5941c08d4cd None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:18.565793 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23951a25-fe3f-4675-817f-a5941c08d4cd None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:18.567248 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23951a25-fe3f-4675-817f-a5941c08d4cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:18.567799 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23951a25-fe3f-4675-817f-a5941c08d4cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:18.570504 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23951a25-fe3f-4675-817f-a5941c08d4cd None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:18.575707 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44044c4e-6de0-4065-bb87-9af4b2393e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:18.577343 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44044c4e-6de0-4065-bb87-9af4b2393e02 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:18.578161 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44044c4e-6de0-4065-bb87-9af4b2393e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:18.578886 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44044c4e-6de0-4065-bb87-9af4b2393e02 None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:18.580590 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44044c4e-6de0-4065-bb87-9af4b2393e02 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:18.581492 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44044c4e-6de0-4065-bb87-9af4b2393e02 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:18.584518 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44044c4e-6de0-4065-bb87-9af4b2393e02 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:18.601953 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 371/758] 10.222.0.87 () {56 vars in 1148 bytes} [Tue Jun 21 18:13:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:18.603705 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23951a25-fe3f-4675-817f-a5941c08d4cd None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:18.613969 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 388/759] 10.222.0.87 () {56 vars in 1148 bytes} [Tue Jun 21 18:13:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:18.616327 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44044c4e-6de0-4065-bb87-9af4b2393e02 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:18.651470 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:18.804785 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44044c4e-6de0-4065-bb87-9af4b2393e02 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:18.820092 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23951a25-fe3f-4675-817f-a5941c08d4cd None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:18.823141 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:18.886309 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33326663-19d6-4414-a09d-7708a4a8464f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:18.886613 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a07672-074f-438f-81cf-d22c3f7f5e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:18.887223 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a07672-074f-438f-81cf-d22c3f7f5e2a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:18.887516 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a07672-074f-438f-81cf-d22c3f7f5e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:18.887796 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a07672-074f-438f-81cf-d22c3f7f5e2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:18.888643 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33326663-19d6-4414-a09d-7708a4a8464f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:18.889133 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33326663-19d6-4414-a09d-7708a4a8464f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:18.889613 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33326663-19d6-4414-a09d-7708a4a8464f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:18.891269 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33326663-19d6-4414-a09d-7708a4a8464f None admin] RBAC: Adding query filter params (role.id=98f0943a610f4a9e9f8de54f6e9f717d, scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb, user.id=515b3571045e4fe29565307c8ecc10b2) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:18.891818 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33326663-19d6-4414-a09d-7708a4a8464f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:18.894656 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33326663-19d6-4414-a09d-7708a4a8464f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:18.905718 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a07672-074f-438f-81cf-d22c3f7f5e2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:18.908535 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a07672-074f-438f-81cf-d22c3f7f5e2a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:18.921113 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 389/760] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:13:18 2022] GET /identity/v3/role_assignments?user.id=515b3571045e4fe29565307c8ecc10b2&scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb&role.id=98f0943a610f4a9e9f8de54f6e9f717d => generated 259 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:18.921113 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33326663-19d6-4414-a09d-7708a4a8464f None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:18.950130 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-e2a07672-074f-438f-81cf-d22c3f7f5e2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:18.950130 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:18.950130 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:18.950130 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:18.950130 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:18.950130 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:18.950130 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:18.950130 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:18.950130 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:18.950130 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:18.950130 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:18.950130 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:18.950130 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:13:18.950130 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:13:18.950130 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:13:18.951829 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:13:18.951829 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:18.951829 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:18.951829 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:18.951829 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:18.951829 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:18.951829 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:18.951829 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:18.951829 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:18.951829 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:18.951829 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:18.951829 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:18.951829 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:18.951829 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:18.951829 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:18.951829 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:18.951829 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:18.953508 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:13:18.953508 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:13:18.953508 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:18.953508 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:18.957328 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 372/761] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:13:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:18.959325 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d14c8bf-1ae5-419c-86f5-b8ecade7be2c None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:18.959717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d14c8bf-1ae5-419c-86f5-b8ecade7be2c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:18.960105 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d14c8bf-1ae5-419c-86f5-b8ecade7be2c None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:18.964039 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 373/762] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:13:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:13:18.965470 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d14c8bf-1ae5-419c-86f5-b8ecade7be2c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:19.065176 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:19.106149 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33326663-19d6-4414-a09d-7708a4a8464f None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:19.175899 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c1beb9-e556-4374-8a18-df6330cfcde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:19.176615 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c1beb9-e556-4374-8a18-df6330cfcde9 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:19.177052 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c1beb9-e556-4374-8a18-df6330cfcde9 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:19.177390 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c1beb9-e556-4374-8a18-df6330cfcde9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:19.178308 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c1beb9-e556-4374-8a18-df6330cfcde9 None admin] RBAC: Adding query filter params (role.id=b9dbf55e6f2947158c1d946f6ee9f907, scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb, user.id=09c9fa7dccb8491c9ad3b43967345ea2) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:19.178629 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c1beb9-e556-4374-8a18-df6330cfcde9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:19.180129 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c1beb9-e556-4374-8a18-df6330cfcde9 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:19.187747 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 390/763] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:13:18 2022] GET /identity/v3/role_assignments?user.id=09c9fa7dccb8491c9ad3b43967345ea2&scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb&role.id=b9dbf55e6f2947158c1d946f6ee9f907 => generated 259 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:19.189139 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee716fc-51a2-41c7-84f3-267c51ca7236 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:19.189440 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee716fc-51a2-41c7-84f3-267c51ca7236 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:19.189739 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee716fc-51a2-41c7-84f3-267c51ca7236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:19.213695 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-5ee716fc-51a2-41c7-84f3-267c51ca7236 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:13:19.215699 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d14c8bf-1ae5-419c-86f5-b8ecade7be2c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:19.216441 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5ee716fc-51a2-41c7-84f3-267c51ca7236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:13:19.284580 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64556d9-a9bf-4217-b444-81eb035d4c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:19.287580 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64556d9-a9bf-4217-b444-81eb035d4c78 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:19.288088 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64556d9-a9bf-4217-b444-81eb035d4c78 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:19.288517 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64556d9-a9bf-4217-b444-81eb035d4c78 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:19.289894 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64556d9-a9bf-4217-b444-81eb035d4c78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:19.290392 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64556d9-a9bf-4217-b444-81eb035d4c78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:19.292735 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64556d9-a9bf-4217-b444-81eb035d4c78 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:19.338205 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 374/764] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:13:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:19.349684 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64556d9-a9bf-4217-b444-81eb035d4c78 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:19.407041 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:19.423566 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 391/765] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:13:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:13:19.538028 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee716fc-51a2-41c7-84f3-267c51ca7236 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:19.672542 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64556d9-a9bf-4217-b444-81eb035d4c78 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:19.748158 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:19.820253 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f967dbfd-0002-493f-ad36-93ee9081afa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:19.822619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f967dbfd-0002-493f-ad36-93ee9081afa0 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:19.823479 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f967dbfd-0002-493f-ad36-93ee9081afa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:19.824233 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f967dbfd-0002-493f-ad36-93ee9081afa0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:19.840195 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f967dbfd-0002-493f-ad36-93ee9081afa0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:19.848863 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f967dbfd-0002-493f-ad36-93ee9081afa0 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:19.854040 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee716fc-51a2-41c7-84f3-267c51ca7236 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:19.876733 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-f967dbfd-0002-493f-ad36-93ee9081afa0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:13:19.876733 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:19.876733 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:19.876733 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:19.876733 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:19.876733 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:19.876733 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:19.876733 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:19.876733 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:19.876733 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:19.876733 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:19.876733 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:19.876733 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:13:19.876733 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:13:19.876733 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:13:19.878050 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:13:19.878050 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:19.878050 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:19.878050 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:19.878050 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:19.878050 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:19.878050 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:19.878050 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:19.878050 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:19.878050 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:19.878050 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:19.878050 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:19.878050 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:19.878050 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:19.878050 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:19.878050 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:19.878050 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:19.879276 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:13:19.879276 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:13:19.879276 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:13:19.879276 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:13:19.879276 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:13:19.879276 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:13:19.879276 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:13:19.879276 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:19.883792 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 375/766] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:13:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 535 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:19.895887 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f967dbfd-0002-493f-ad36-93ee9081afa0 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:19.957736 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f69a85-82a3-4e8a-9d6a-1f640590e944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:19.958615 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f69a85-82a3-4e8a-9d6a-1f640590e944 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:19.958934 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f69a85-82a3-4e8a-9d6a-1f640590e944 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:19.959176 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f69a85-82a3-4e8a-9d6a-1f640590e944 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:19.960265 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f69a85-82a3-4e8a-9d6a-1f640590e944 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:19.962682 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f69a85-82a3-4e8a-9d6a-1f640590e944 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:19.991341 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-c3f69a85-82a3-4e8a-9d6a-1f640590e944 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 21 18:13:19.991341 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:19.991341 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:13:19.991341 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:13:19.991341 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:13:19.991341 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:13:19.991341 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:13:19.991341 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:13:19.991341 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 21 18:13:19.991341 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:19.991341 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:13:19.991341 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:19.991341 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:19.991341 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:19.991341 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:19.991341 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:19.993611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:19.993611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:19.993611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:19.993611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:19.993611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:19.993611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:19.993611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:19.993611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:13:19.993611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:13:19.993611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:13:19.993611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:13:19.993611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:19.993611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:19.993611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:19.993611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:19.993611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:19.996443 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:19.996443 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:19.996443 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:19.996443 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:19.996443 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:19.996443 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:19.996443 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:19.996443 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:19.996443 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:19.996443 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:19.996443 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:19.996443 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:13:19.996443 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:13:19.996443 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 21 18:13:19.996443 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:19.996443 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 392/767] 10.222.0.87 () {56 vars in 1129 bytes} [Tue Jun 21 18:13:19 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 458 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:20.011454 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f69a85-82a3-4e8a-9d6a-1f640590e944 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:20.153514 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:20.214123 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f69a85-82a3-4e8a-9d6a-1f640590e944 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:20.264690 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f967dbfd-0002-493f-ad36-93ee9081afa0 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:20.273794 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986fe68b-bf6b-42a9-9cca-75a65fb4b171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:20.275225 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986fe68b-bf6b-42a9-9cca-75a65fb4b171 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:20.276013 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986fe68b-bf6b-42a9-9cca-75a65fb4b171 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:20.277219 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986fe68b-bf6b-42a9-9cca-75a65fb4b171 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:20.280064 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986fe68b-bf6b-42a9-9cca-75a65fb4b171 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:20.281157 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986fe68b-bf6b-42a9-9cca-75a65fb4b171 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:20.283125 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986fe68b-bf6b-42a9-9cca-75a65fb4b171 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:20.305839 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 393/768] 10.222.0.87 () {56 vars in 1124 bytes} [Tue Jun 21 18:13:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:20.308802 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986fe68b-bf6b-42a9-9cca-75a65fb4b171 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:20.315985 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:20.403998 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89467a00-c54f-479f-bebf-aa675133a9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:20.405124 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89467a00-c54f-479f-bebf-aa675133a9dc None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:20.405624 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89467a00-c54f-479f-bebf-aa675133a9dc None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:20.406140 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89467a00-c54f-479f-bebf-aa675133a9dc None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:20.407678 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89467a00-c54f-479f-bebf-aa675133a9dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:20.408241 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89467a00-c54f-479f-bebf-aa675133a9dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:20.411082 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89467a00-c54f-479f-bebf-aa675133a9dc None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:20.449227 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 376/769] 10.222.0.87 () {56 vars in 1148 bytes} [Tue Jun 21 18:13:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 554 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:20.451487 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e4b552-fef1-438f-9113-8dadf6d9a286 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:20.452063 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e4b552-fef1-438f-9113-8dadf6d9a286 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:20.452550 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e4b552-fef1-438f-9113-8dadf6d9a286 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:20.455254 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 377/770] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:13:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:13:20.464993 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f0b97-c14d-4aab-8c8b-c87d709910f8 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:20.466119 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f0b97-c14d-4aab-8c8b-c87d709910f8 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:20.466870 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f0b97-c14d-4aab-8c8b-c87d709910f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:20.483045 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986fe68b-bf6b-42a9-9cca-75a65fb4b171 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:20.496884 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b10f0b97-c14d-4aab-8c8b-c87d709910f8 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:13:20.499839 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b10f0b97-c14d-4aab-8c8b-c87d709910f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:13:20.528701 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1bfa02-c48f-4de4-8740-c756efcec29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:20.529520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1bfa02-c48f-4de4-8740-c756efcec29b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:20.529847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1bfa02-c48f-4de4-8740-c756efcec29b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:20.530132 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1bfa02-c48f-4de4-8740-c756efcec29b None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:20.531152 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1bfa02-c48f-4de4-8740-c756efcec29b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:20.531475 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1bfa02-c48f-4de4-8740-c756efcec29b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:20.533600 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1bfa02-c48f-4de4-8740-c756efcec29b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:20.549481 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 394/771] 10.222.0.87 () {56 vars in 1124 bytes} [Tue Jun 21 18:13:20 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:13:20.549481 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1bfa02-c48f-4de4-8740-c756efcec29b None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:20.651377 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:20.668793 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 378/772] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:13:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:13:20.671008 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f0b97-c14d-4aab-8c8b-c87d709910f8 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:20.709844 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1bfa02-c48f-4de4-8740-c756efcec29b None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:20.766964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d09907b-926d-42ba-bed4-355509356c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:20.768209 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09907b-926d-42ba-bed4-355509356c37 None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:20.768209 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09907b-926d-42ba-bed4-355509356c37 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:20.768209 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09907b-926d-42ba-bed4-355509356c37 None admin] PATH_INFO: `/v3/projects/5f30c3ddb5e44e9299ff228fe94c27fb/users/a1b3fa67905c4027b3656ac36e2b76c9/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:20.771423 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d09907b-926d-42ba-bed4-355509356c37 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f30c3ddb5e44e9299ff228fe94c27fb, user_id=a1b3fa67905c4027b3656ac36e2b76c9, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:20.774933 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d09907b-926d-42ba-bed4-355509356c37 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:20.827069 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 395/773] 10.222.0.87 () {58 vars in 1439 bytes} [Tue Jun 21 18:13:20 2022] PUT /identity/v3/projects/5f30c3ddb5e44e9299ff228fe94c27fb/users/a1b3fa67905c4027b3656ac36e2b76c9/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:13:20.827069 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d09907b-926d-42ba-bed4-355509356c37 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:20.853677 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:20.966697 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f0b97-c14d-4aab-8c8b-c87d709910f8 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:21.054583 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd54905-605c-41d9-9bcd-3692b1c4b57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:21.057277 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd54905-605c-41d9-9bcd-3692b1c4b57f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:21.059956 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd54905-605c-41d9-9bcd-3692b1c4b57f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:21.060836 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd54905-605c-41d9-9bcd-3692b1c4b57f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:21.063308 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd54905-605c-41d9-9bcd-3692b1c4b57f None admin] RBAC: Adding query filter params (interface=public, service_id=032989e0f47048948cf0c601610ad760, region_id=RegionOne) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:21.067557 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd54905-605c-41d9-9bcd-3692b1c4b57f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:21.074521 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd54905-605c-41d9-9bcd-3692b1c4b57f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:21.097787 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 379/774] 10.222.0.87 () {56 vars in 1251 bytes} [Tue Jun 21 18:13:20 2022] GET /identity/v3/endpoints?service_id=032989e0f47048948cf0c601610ad760&interface=public®ion_id=RegionOne => generated 194 bytes in 426 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:21.105518 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd54905-605c-41d9-9bcd-3692b1c4b57f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:21.114004 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d09907b-926d-42ba-bed4-355509356c37 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:21.155272 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:21.181037 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb71713e-f0f6-498d-b89d-4e75bfe3c988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:21.182384 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb71713e-f0f6-498d-b89d-4e75bfe3c988 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:21.182853 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb71713e-f0f6-498d-b89d-4e75bfe3c988 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:21.183313 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb71713e-f0f6-498d-b89d-4e75bfe3c988 None admin] PATH_INFO: `/v3/services/network` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:21.184509 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb71713e-f0f6-498d-b89d-4e75bfe3c988 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:21.187049 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb71713e-f0f6-498d-b89d-4e75bfe3c988 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:21.205943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-cb71713e-f0f6-498d-b89d-4e75bfe3c988 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 21 18:13:21.205943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:21.205943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:13:21.205943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:13:21.205943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:13:21.205943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:13:21.205943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:13:21.205943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:13:21.205943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 21 18:13:21.205943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:21.205943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:13:21.205943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:21.205943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:21.205943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:21.205943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:21.205943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:21.205943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:21.207921 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:21.207921 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:21.207921 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:21.207921 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:21.207921 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:21.207921 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:21.207921 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:13:21.207921 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:13:21.207921 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:13:21.207921 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:13:21.207921 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:21.207921 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:21.207921 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:21.207921 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:21.207921 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:21.207921 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:21.210130 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:21.210130 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:21.210130 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:21.210130 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:21.210130 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:21.210130 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:21.210130 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:21.210130 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:21.210130 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:21.210130 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:21.210130 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:13:21.210130 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:13:21.210130 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 21 18:13:21.210130 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:21.210130 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 396/775] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:13:20 2022] GET /identity/v3/services/network => generated 88 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:21.218868 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb71713e-f0f6-498d-b89d-4e75bfe3c988 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:21.339021 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd54905-605c-41d9-9bcd-3692b1c4b57f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:21.388086 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb71713e-f0f6-498d-b89d-4e75bfe3c988 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:21.395093 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:21.410547 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad2c736-6734-41d5-8eaf-fd8bc5913397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:21.412569 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2c736-6734-41d5-8eaf-fd8bc5913397 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:21.413437 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2c736-6734-41d5-8eaf-fd8bc5913397 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:21.414208 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2c736-6734-41d5-8eaf-fd8bc5913397 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:21.415973 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad2c736-6734-41d5-8eaf-fd8bc5913397 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:21.418978 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad2c736-6734-41d5-8eaf-fd8bc5913397 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:21.434075 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 380/776] 10.222.0.87 () {56 vars in 1087 bytes} [Tue Jun 21 18:13:21 2022] GET /identity/v3/services => generated 772 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:21.474312 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f13a60-6659-4952-be5e-a0b0ee5197f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:21.475028 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f13a60-6659-4952-be5e-a0b0ee5197f9 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:21.475321 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f13a60-6659-4952-be5e-a0b0ee5197f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:21.475604 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f13a60-6659-4952-be5e-a0b0ee5197f9 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:21.476362 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f13a60-6659-4952-be5e-a0b0ee5197f9 None admin] RBAC: Adding query filter params (name=network) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:21.476680 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f13a60-6659-4952-be5e-a0b0ee5197f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:21.478283 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f13a60-6659-4952-be5e-a0b0ee5197f9 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:21.484487 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 397/777] 10.222.0.87 () {56 vars in 1112 bytes} [Tue Jun 21 18:13:21 2022] GET /identity/v3/services?name=network => generated 124 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:13:21.493703 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad2c736-6734-41d5-8eaf-fd8bc5913397 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:21.714211 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad2c736-6734-41d5-8eaf-fd8bc5913397 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:21.781158 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df1f63f-d9d0-4133-a143-3a4a3d065b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:21.782859 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df1f63f-d9d0-4133-a143-3a4a3d065b37 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:21.783686 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df1f63f-d9d0-4133-a143-3a4a3d065b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:21.784525 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df1f63f-d9d0-4133-a143-3a4a3d065b37 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:21.786292 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df1f63f-d9d0-4133-a143-3a4a3d065b37 None admin] RBAC: Adding query filter params (type=network) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:21.787268 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df1f63f-d9d0-4133-a143-3a4a3d065b37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:21.790143 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df1f63f-d9d0-4133-a143-3a4a3d065b37 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:21.803554 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 381/778] 10.222.0.87 () {56 vars in 1112 bytes} [Tue Jun 21 18:13:21 2022] GET /identity/v3/services?type=network => generated 350 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:21.814408 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f13a60-6659-4952-be5e-a0b0ee5197f9 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:21.936343 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:22.036292 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f13a60-6659-4952-be5e-a0b0ee5197f9 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:22.134888 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3dbb09-6bb0-438d-89dd-ab877f1244f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:22.136028 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3dbb09-6bb0-438d-89dd-ab877f1244f4 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:22.136529 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3dbb09-6bb0-438d-89dd-ab877f1244f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:22.137014 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3dbb09-6bb0-438d-89dd-ab877f1244f4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:22.137968 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3dbb09-6bb0-438d-89dd-ab877f1244f4 None admin] RBAC: Adding query filter params (interface=public, service_id=5ba6b38003674c25ab1b70d8aed1ff33, region_id=RegionOne) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:22.138348 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3dbb09-6bb0-438d-89dd-ab877f1244f4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:22.140439 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3dbb09-6bb0-438d-89dd-ab877f1244f4 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:22.150897 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 398/779] 10.222.0.87 () {56 vars in 1251 bytes} [Tue Jun 21 18:13:21 2022] GET /identity/v3/endpoints?service_id=5ba6b38003674c25ab1b70d8aed1ff33&interface=public®ion_id=RegionOne => generated 194 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:22.158967 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df1f63f-d9d0-4133-a143-3a4a3d065b37 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:22.271700 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:22.340883 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df1f63f-d9d0-4133-a143-3a4a3d065b37 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:22.391758 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de687867-7c10-4c46-b04a-8210e066acd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:22.392462 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de687867-7c10-4c46-b04a-8210e066acd2 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:22.392747 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de687867-7c10-4c46-b04a-8210e066acd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:22.393022 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de687867-7c10-4c46-b04a-8210e066acd2 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:22.393776 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de687867-7c10-4c46-b04a-8210e066acd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:22.395200 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de687867-7c10-4c46-b04a-8210e066acd2 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:22.402570 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 382/780] 10.222.0.87 () {56 vars in 1087 bytes} [Tue Jun 21 18:13:22 2022] GET /identity/v3/services => generated 772 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:22.456316 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8102d-f33d-4a19-8ce3-f5cc189f24b6 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:22.456799 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8102d-f33d-4a19-8ce3-f5cc189f24b6 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:22.457280 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8102d-f33d-4a19-8ce3-f5cc189f24b6 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:22.459968 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 399/781] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:13:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:13:22.467749 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec90aaa-1dfa-4c74-954a-341f8c3f126b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:22.468436 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec90aaa-1dfa-4c74-954a-341f8c3f126b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:22.468914 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec90aaa-1dfa-4c74-954a-341f8c3f126b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:22.499669 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-8ec90aaa-1dfa-4c74-954a-341f8c3f126b None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:13:22.501726 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8ec90aaa-1dfa-4c74-954a-341f8c3f126b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:13:22.596225 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 383/782] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:13:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:13:22.641597 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e8102d-f33d-4a19-8ce3-f5cc189f24b6 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:22.850338 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e8102d-f33d-4a19-8ce3-f5cc189f24b6 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:22.857689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7fae3-abf9-4b53-83a4-4322669d3830 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:22.858125 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7fae3-abf9-4b53-83a4-4322669d3830 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:22.858607 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7fae3-abf9-4b53-83a4-4322669d3830 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:22.861404 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 384/783] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:13:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:13:22.868870 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d16a11-8880-40ce-8864-3847827729c7 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:22.869863 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d16a11-8880-40ce-8864-3847827729c7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:22.870696 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d16a11-8880-40ce-8864-3847827729c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:22.903586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:22.919265 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-82d16a11-8880-40ce-8864-3847827729c7 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:13:22.927398 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-82d16a11-8880-40ce-8864-3847827729c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:13:22.952760 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed62fdab-5f6c-47f6-a4ba-ca490e132dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:22.954740 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62fdab-5f6c-47f6-a4ba-ca490e132dab None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:22.955606 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62fdab-5f6c-47f6-a4ba-ca490e132dab None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:22.956385 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62fdab-5f6c-47f6-a4ba-ca490e132dab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:22.997204 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed62fdab-5f6c-47f6-a4ba-ca490e132dab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:13:23.023351 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-ed62fdab-5f6c-47f6-a4ba-ca490e132dab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:13:23.023351 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:23.023351 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:23.023351 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:23.023351 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:23.023351 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:23.023351 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:23.023351 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:23.023351 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:23.023351 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:23.023351 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:23.023351 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:23.023351 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:13:23.023351 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:13:23.023351 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:13:23.023351 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:13:23.025314 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:13:23.025314 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:13:23.025314 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:13:23.025314 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:13:23.025314 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:23.025314 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:23.025314 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:23.025314 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:23.025314 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:23.025314 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:23.025314 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:23.025314 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:23.025314 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:23.025314 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:23.025314 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:23.025314 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:23.026990 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:23.026990 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:23.026990 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:23.026990 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:23.026990 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:13:23.026990 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:13:23.026990 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:13:23.026990 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:13:23.026990 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:13:23.026990 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:13:23.026990 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:13:23.026990 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:23.031878 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 400/784] 10.222.0.87 () {56 vars in 1096 bytes} [Tue Jun 21 18:13:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 391 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:23.042526 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed62fdab-5f6c-47f6-a4ba-ca490e132dab None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:23.094099 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:23.139671 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 385/785] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:13:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:13:23.229279 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d16a11-8880-40ce-8864-3847827729c7 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:23.285007 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed62fdab-5f6c-47f6-a4ba-ca490e132dab None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:23.349750 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5544aa0-df57-41b6-b7fa-46daf0bbdfe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:23.350856 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5544aa0-df57-41b6-b7fa-46daf0bbdfe6 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:23.351372 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5544aa0-df57-41b6-b7fa-46daf0bbdfe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:23.351913 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5544aa0-df57-41b6-b7fa-46daf0bbdfe6 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:23.353472 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5544aa0-df57-41b6-b7fa-46daf0bbdfe6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:23.354376 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5544aa0-df57-41b6-b7fa-46daf0bbdfe6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:23.356923 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5544aa0-df57-41b6-b7fa-46daf0bbdfe6 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:23.371545 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 401/786] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:13:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:13:23.379794 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5544aa0-df57-41b6-b7fa-46daf0bbdfe6 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:23.442087 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:23.520318 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5544aa0-df57-41b6-b7fa-46daf0bbdfe6 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:23.570724 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d6ec9-d897-43a3-9807-85b950049b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:23.572437 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d6ec9-d897-43a3-9807-85b950049b4a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:23.573029 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d6ec9-d897-43a3-9807-85b950049b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:23.573584 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d6ec9-d897-43a3-9807-85b950049b4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:23.591866 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0d6ec9-d897-43a3-9807-85b950049b4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:23.594292 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0d6ec9-d897-43a3-9807-85b950049b4a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:23.609240 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d16a11-8880-40ce-8864-3847827729c7 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:23.623152 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-aa0d6ec9-d897-43a3-9807-85b950049b4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:23.623152 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:23.623152 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:23.623152 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:23.623152 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:23.623152 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:23.623152 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:23.623152 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:23.623152 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:23.623152 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:23.623152 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:23.623152 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:23.623152 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:13:23.623152 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:13:23.623152 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:13:23.624276 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:13:23.624276 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:23.624276 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:23.624276 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:23.624276 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:23.624276 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:23.624276 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:23.624276 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:23.624276 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:23.624276 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:23.624276 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:23.624276 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:23.624276 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:23.624276 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:23.624276 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:23.624276 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:23.624276 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:23.625495 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:13:23.625495 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:13:23.625495 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:23.625495 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:23.628722 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 402/787] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:13:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:13:23.636475 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d6ec9-d897-43a3-9807-85b950049b4a None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:23.708105 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b8ba9b-c92b-42d3-af10-d8a9371be2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:23.709622 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8ba9b-c92b-42d3-af10-d8a9371be2e8 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:23.710096 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8ba9b-c92b-42d3-af10-d8a9371be2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:23.710564 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8ba9b-c92b-42d3-af10-d8a9371be2e8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:23.742319 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b8ba9b-c92b-42d3-af10-d8a9371be2e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:13:23.751945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-b9b8ba9b-c92b-42d3-af10-d8a9371be2e8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:13:23.751945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:23.751945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:23.751945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:23.751945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:23.751945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:23.751945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:23.751945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:23.751945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:23.751945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:23.751945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:23.751945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:23.751945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:13:23.751945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:13:23.751945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:13:23.751945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:13:23.752982 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:13:23.752982 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:13:23.752982 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:13:23.752982 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:13:23.752982 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:23.752982 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:23.752982 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:23.752982 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:23.752982 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:23.752982 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:23.752982 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:23.752982 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:23.752982 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:23.752982 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:23.752982 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:23.752982 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:23.753987 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:23.753987 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:23.753987 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:23.753987 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:23.753987 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:13:23.753987 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:13:23.753987 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:13:23.753987 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:13:23.753987 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:13:23.753987 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:13:23.753987 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:13:23.753987 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:23.765680 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 386/788] 10.222.0.87 () {56 vars in 1102 bytes} [Tue Jun 21 18:13:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 533 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:23.780298 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b8ba9b-c92b-42d3-af10-d8a9371be2e8 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:23.793576 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:23.811080 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d6ec9-d897-43a3-9807-85b950049b4a None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:23.884438 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc737de-9883-47e7-b477-0ce1024acbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:23.887024 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc737de-9883-47e7-b477-0ce1024acbdd None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:23.890774 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc737de-9883-47e7-b477-0ce1024acbdd None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:23.891732 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc737de-9883-47e7-b477-0ce1024acbdd None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:23.893467 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc737de-9883-47e7-b477-0ce1024acbdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:23.896477 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc737de-9883-47e7-b477-0ce1024acbdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:23.900619 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc737de-9883-47e7-b477-0ce1024acbdd None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:23.942944 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 403/789] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:13:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:23.953045 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc737de-9883-47e7-b477-0ce1024acbdd None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:23.955426 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:24.204641 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc737de-9883-47e7-b477-0ce1024acbdd None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:24.273176 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b8ba9b-c92b-42d3-af10-d8a9371be2e8 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:24.296681 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442b4fa6-d27a-4e6e-8b8c-39754eeb8411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:24.298911 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b4fa6-d27a-4e6e-8b8c-39754eeb8411 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:24.299811 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b4fa6-d27a-4e6e-8b8c-39754eeb8411 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:24.300577 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b4fa6-d27a-4e6e-8b8c-39754eeb8411 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:24.322013 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442b4fa6-d27a-4e6e-8b8c-39754eeb8411 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:24.325459 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442b4fa6-d27a-4e6e-8b8c-39754eeb8411 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:24.376512 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-442b4fa6-d27a-4e6e-8b8c-39754eeb8411 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 21 18:13:24.376512 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:24.376512 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:24.376512 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:24.376512 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:24.376512 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:24.376512 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:24.376512 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:24.376512 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:24.376512 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:24.376512 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:24.376512 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:24.376512 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:13:24.376512 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:13:24.376512 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:13:24.376512 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:13:24.378345 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:24.378345 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:24.378345 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:13:24.378345 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:13:24.378345 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:13:24.378345 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:13:24.378345 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:24.378345 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:24.378345 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:24.378345 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:24.378345 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:24.378345 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:24.378345 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:24.378345 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:24.378345 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:24.378345 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:24.378345 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:24.378345 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:24.380100 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:24.380100 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:24.380100 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:13:24.380100 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:13:24.380100 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:13:24.380100 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:13:24.380100 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:13:24.380100 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:13:24.380100 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 21 18:13:24.380100 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:24.380100 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 404/790] 10.222.0.87 () {56 vars in 1096 bytes} [Tue Jun 21 18:13:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 427 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:13:24.389360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442b4fa6-d27a-4e6e-8b8c-39754eeb8411 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:24.420082 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:24.433325 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd92c11-2f58-45b7-877b-dff242112863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:24.434648 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd92c11-2f58-45b7-877b-dff242112863 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:24.435118 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd92c11-2f58-45b7-877b-dff242112863 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:24.435586 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd92c11-2f58-45b7-877b-dff242112863 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:24.436928 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd92c11-2f58-45b7-877b-dff242112863 None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:24.437449 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd92c11-2f58-45b7-877b-dff242112863 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:24.439917 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd92c11-2f58-45b7-877b-dff242112863 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:24.473380 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 387/791] 10.222.0.87 () {56 vars in 1103 bytes} [Tue Jun 21 18:13:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 693 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:13:24.483433 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd92c11-2f58-45b7-877b-dff242112863 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:24.622496 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442b4fa6-d27a-4e6e-8b8c-39754eeb8411 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:24.700751 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690af667-7a50-4da0-8ff2-e52cc823f3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:24.702176 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690af667-7a50-4da0-8ff2-e52cc823f3ab None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:24.702708 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690af667-7a50-4da0-8ff2-e52cc823f3ab None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:24.703204 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690af667-7a50-4da0-8ff2-e52cc823f3ab None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:24.705113 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690af667-7a50-4da0-8ff2-e52cc823f3ab None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:24.705871 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690af667-7a50-4da0-8ff2-e52cc823f3ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:24.708802 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690af667-7a50-4da0-8ff2-e52cc823f3ab None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:24.747480 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 405/792] 10.222.0.87 () {56 vars in 1135 bytes} [Tue Jun 21 18:13:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:24.750143 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed80bd-0260-429f-8b6d-bd1af4247e41 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:24.750641 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed80bd-0260-429f-8b6d-bd1af4247e41 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:24.751136 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed80bd-0260-429f-8b6d-bd1af4247e41 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:24.754536 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 406/793] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:13:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:13:24.756003 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ed80bd-0260-429f-8b6d-bd1af4247e41 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:24.764884 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd92c11-2f58-45b7-877b-dff242112863 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:24.768138 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:24.823954 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9d8d99-b742-4c3a-8c40-5102d1b1dcd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:24.825549 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d8d99-b742-4c3a-8c40-5102d1b1dcd4 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:24.826024 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d8d99-b742-4c3a-8c40-5102d1b1dcd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:24.826494 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d8d99-b742-4c3a-8c40-5102d1b1dcd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:24.846583 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9d8d99-b742-4c3a-8c40-5102d1b1dcd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:24.848746 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9d8d99-b742-4c3a-8c40-5102d1b1dcd4 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:24.878301 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-5f9d8d99-b742-4c3a-8c40-5102d1b1dcd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:24.878301 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:24.878301 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:24.878301 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:24.878301 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:24.878301 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:24.878301 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:24.878301 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:24.878301 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:24.878301 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:24.878301 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:24.878301 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:24.878301 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:13:24.878301 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:13:24.878301 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:13:24.879974 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:13:24.879974 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:24.879974 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:24.879974 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:24.879974 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:24.879974 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:24.879974 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:24.879974 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:24.879974 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:24.879974 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:24.879974 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:24.879974 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:24.879974 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:24.879974 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:24.879974 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:24.879974 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:24.879974 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:24.881438 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:13:24.881438 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:13:24.881438 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:24.881438 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:24.884342 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 388/794] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:13:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 402 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:13:24.887102 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e858a35-e1a1-4796-864a-6d13ef08511e None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:24.887908 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e858a35-e1a1-4796-864a-6d13ef08511e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:24.888627 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e858a35-e1a1-4796-864a-6d13ef08511e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:24.919210 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3e858a35-e1a1-4796-864a-6d13ef08511e None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:13:24.921187 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3e858a35-e1a1-4796-864a-6d13ef08511e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:13:24.923621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:25.002901 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ed80bd-0260-429f-8b6d-bd1af4247e41 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:25.041819 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 389/795] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:13:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:13:25.043345 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e858a35-e1a1-4796-864a-6d13ef08511e None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:25.082591 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c05360-3cd2-41a8-80c2-1645989304d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:25.084057 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c05360-3cd2-41a8-80c2-1645989304d7 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:25.084704 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c05360-3cd2-41a8-80c2-1645989304d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:25.085079 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c05360-3cd2-41a8-80c2-1645989304d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:25.109484 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c05360-3cd2-41a8-80c2-1645989304d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:25.117611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c05360-3cd2-41a8-80c2-1645989304d7 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:25.152676 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-34c05360-3cd2-41a8-80c2-1645989304d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:25.152676 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:25.152676 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:25.152676 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:25.152676 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:25.152676 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:25.152676 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:25.152676 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:25.152676 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:25.152676 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:25.152676 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:25.152676 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:25.152676 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:13:25.152676 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:13:25.152676 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:13:25.159669 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:13:25.159669 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:25.159669 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:25.159669 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:25.159669 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:25.159669 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:25.159669 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:25.159669 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:25.159669 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:25.159669 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:25.159669 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:25.159669 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:25.159669 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:25.159669 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:25.159669 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:25.159669 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:25.159669 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:25.161492 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:13:25.161492 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:13:25.161492 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:25.161492 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:25.161492 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 407/796] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:13:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 400 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:25.161492 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f22af-ed5f-41c4-b7d8-a4a52c722acf None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:25.161492 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f22af-ed5f-41c4-b7d8-a4a52c722acf None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:25.161492 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f22af-ed5f-41c4-b7d8-a4a52c722acf None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:25.167249 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 408/797] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:13:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:13:25.167249 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f22af-ed5f-41c4-b7d8-a4a52c722acf None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:25.254836 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:25.299362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e858a35-e1a1-4796-864a-6d13ef08511e None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:25.384490 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b02946-16dd-4fbb-a928-e7734bb09075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:25.385942 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b02946-16dd-4fbb-a928-e7734bb09075 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:25.387256 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b02946-16dd-4fbb-a928-e7734bb09075 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:25.387554 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b02946-16dd-4fbb-a928-e7734bb09075 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:25.389049 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b02946-16dd-4fbb-a928-e7734bb09075 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:25.389868 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b02946-16dd-4fbb-a928-e7734bb09075 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:25.393160 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b02946-16dd-4fbb-a928-e7734bb09075 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:25.413337 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f22af-ed5f-41c4-b7d8-a4a52c722acf None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:25.421087 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 390/798] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:13:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:25.422187 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b02946-16dd-4fbb-a928-e7734bb09075 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:25.473193 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:25.501365 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e650861b-7184-4495-8a8f-1bdabd960b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:25.502848 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650861b-7184-4495-8a8f-1bdabd960b09 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:25.503365 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650861b-7184-4495-8a8f-1bdabd960b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:25.503858 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650861b-7184-4495-8a8f-1bdabd960b09 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:25.526788 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e650861b-7184-4495-8a8f-1bdabd960b09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:13:25.542258 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-e650861b-7184-4495-8a8f-1bdabd960b09 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:13:25.542258 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:25.542258 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:25.542258 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:25.542258 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:25.542258 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:25.542258 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:25.542258 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:25.542258 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:25.542258 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:25.542258 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:25.542258 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:25.542258 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:13:25.542258 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:13:25.542258 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:13:25.542258 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:13:25.543950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:13:25.543950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:13:25.543950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:13:25.543950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:13:25.543950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:25.543950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:25.543950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:25.543950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:25.543950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:25.543950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:25.543950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:25.543950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:25.543950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:25.543950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:25.543950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:25.543950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:25.545476 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:25.545476 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:25.545476 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:25.545476 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:25.545476 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:13:25.545476 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:13:25.545476 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:13:25.545476 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:13:25.545476 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:13:25.545476 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:13:25.545476 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:13:25.545476 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:25.545476 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 409/799] 10.222.0.87 () {56 vars in 1096 bytes} [Tue Jun 21 18:13:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 381 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:25.547064 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ac45f-7c78-4321-bb52-274dc7e39d4d None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:25.547064 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ac45f-7c78-4321-bb52-274dc7e39d4d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:25.547963 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ac45f-7c78-4321-bb52-274dc7e39d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:25.583766 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-bc3ac45f-7c78-4321-bb52-274dc7e39d4d None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:13:25.585067 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-bc3ac45f-7c78-4321-bb52-274dc7e39d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:13:25.640060 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b02946-16dd-4fbb-a928-e7734bb09075 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:25.715060 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b66de-a610-4842-bf95-d8fb5b0123ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:25.716230 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b66de-a610-4842-bf95-d8fb5b0123ab None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:25.716736 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b66de-a610-4842-bf95-d8fb5b0123ab None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:25.717230 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b66de-a610-4842-bf95-d8fb5b0123ab None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:25.718746 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b66de-a610-4842-bf95-d8fb5b0123ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:25.719545 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b66de-a610-4842-bf95-d8fb5b0123ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:25.722361 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b66de-a610-4842-bf95-d8fb5b0123ab None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:25.729720 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 410/800] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:13:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:13:25.729720 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ac45f-7c78-4321-bb52-274dc7e39d4d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:25.758961 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 391/801] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:13:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:25.761251 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b66de-a610-4842-bf95-d8fb5b0123ab None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:25.771211 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:25.908992 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ac45f-7c78-4321-bb52-274dc7e39d4d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:25.927750 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:25.968417 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b66de-a610-4842-bf95-d8fb5b0123ab None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:25.968705 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cdbcd2-330a-4c40-ba11-0e601fa7e3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:25.969981 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cdbcd2-330a-4c40-ba11-0e601fa7e3cc None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:25.970499 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cdbcd2-330a-4c40-ba11-0e601fa7e3cc None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:25.970993 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cdbcd2-330a-4c40-ba11-0e601fa7e3cc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:25.985588 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cdbcd2-330a-4c40-ba11-0e601fa7e3cc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:25.988475 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cdbcd2-330a-4c40-ba11-0e601fa7e3cc None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:26.018764 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-e1cdbcd2-330a-4c40-ba11-0e601fa7e3cc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 18:13:26.018764 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:26.018764 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:26.018764 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:26.018764 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:26.018764 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:26.018764 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:26.018764 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:26.018764 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:26.018764 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:26.018764 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:26.018764 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:26.018764 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:13:26.018764 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:13:26.018764 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:13:26.018764 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:13:26.020389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:26.020389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:26.020389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:13:26.020389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:13:26.020389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:13:26.020389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:13:26.020389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:26.020389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:26.020389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:26.020389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:26.020389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:26.020389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:26.020389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:26.020389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:26.020389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:26.020389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:26.020389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:26.020389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:26.022119 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:26.022119 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:26.022119 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:13:26.022119 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:13:26.022119 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:13:26.022119 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:13:26.022119 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:13:26.022119 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:13:26.022119 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 18:13:26.022119 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:26.022119 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 411/802] 10.222.0.87 () {56 vars in 1117 bytes} [Tue Jun 21 18:13:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:26.023558 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cdbcd2-330a-4c40-ba11-0e601fa7e3cc None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:26.028918 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38240a27-ead9-47ca-b8b7-5f52c109f237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:26.030096 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38240a27-ead9-47ca-b8b7-5f52c109f237 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:26.030554 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38240a27-ead9-47ca-b8b7-5f52c109f237 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:26.031045 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38240a27-ead9-47ca-b8b7-5f52c109f237 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:26.032467 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38240a27-ead9-47ca-b8b7-5f52c109f237 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:26.032932 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38240a27-ead9-47ca-b8b7-5f52c109f237 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:26.035592 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38240a27-ead9-47ca-b8b7-5f52c109f237 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:26.053397 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 392/803] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:13:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:26.054810 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38240a27-ead9-47ca-b8b7-5f52c109f237 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:26.178380 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cdbcd2-330a-4c40-ba11-0e601fa7e3cc None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:26.181669 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:26.227627 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38240a27-ead9-47ca-b8b7-5f52c109f237 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:26.237525 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32753438-c466-4cb2-b28e-3bc437a84b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:26.238623 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32753438-c466-4cb2-b28e-3bc437a84b6f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:26.238899 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32753438-c466-4cb2-b28e-3bc437a84b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:26.239181 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32753438-c466-4cb2-b28e-3bc437a84b6f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:26.239853 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32753438-c466-4cb2-b28e-3bc437a84b6f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:26.241264 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32753438-c466-4cb2-b28e-3bc437a84b6f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:26.262200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-32753438-c466-4cb2-b28e-3bc437a84b6f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 21 18:13:26.262200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:26.262200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:13:26.262200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:13:26.262200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:13:26.262200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:13:26.262200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:13:26.262200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:13:26.262200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 21 18:13:26.262200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:26.262200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:13:26.262200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:26.262200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:26.262200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:26.262200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:26.262200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:26.264274 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:26.264274 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:26.264274 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:26.264274 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:26.264274 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:26.264274 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:26.264274 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:26.264274 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:13:26.264274 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:13:26.264274 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:13:26.264274 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:13:26.264274 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:26.264274 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:26.264274 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:26.264274 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:26.264274 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:26.266278 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:26.266278 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:26.266278 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:26.266278 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:26.266278 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:26.266278 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:26.266278 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:26.266278 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:26.266278 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:26.266278 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:26.266278 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:26.266278 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:13:26.266278 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:13:26.266278 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 21 18:13:26.266278 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:26.266278 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 412/804] 10.222.0.87 () {56 vars in 1129 bytes} [Tue Jun 21 18:13:26 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:26.267909 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daacc30-3117-42bf-9234-e2d7c6437c4b None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:26.267909 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daacc30-3117-42bf-9234-e2d7c6437c4b None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:26.267909 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daacc30-3117-42bf-9234-e2d7c6437c4b None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:26.269508 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 413/805] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:13:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:13:26.269731 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daacc30-3117-42bf-9234-e2d7c6437c4b None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:26.290968 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25a18d2-14db-43ad-b963-ea90ef00aab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:26.292193 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a18d2-14db-43ad-b963-ea90ef00aab4 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:26.292668 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a18d2-14db-43ad-b963-ea90ef00aab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:26.293146 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a18d2-14db-43ad-b963-ea90ef00aab4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:26.308870 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25a18d2-14db-43ad-b963-ea90ef00aab4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:26.311812 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25a18d2-14db-43ad-b963-ea90ef00aab4 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:26.324211 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:26.356064 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-a25a18d2-14db-43ad-b963-ea90ef00aab4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:13:26.356064 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:26.356064 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:26.356064 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:26.356064 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:26.356064 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:26.356064 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:26.356064 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:26.356064 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:26.356064 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:26.356064 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:26.356064 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:26.356064 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:13:26.356064 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:13:26.356064 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:13:26.357940 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:13:26.357940 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:26.357940 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:26.357940 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:26.357940 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:26.357940 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:26.357940 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:26.357940 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:26.357940 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:26.357940 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:26.357940 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:26.357940 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:26.357940 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:26.357940 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:26.357940 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:26.357940 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:26.357940 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:26.360543 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:13:26.360543 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:13:26.360543 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:13:26.360543 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:13:26.360543 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:13:26.360543 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:13:26.360543 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:13:26.360543 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:26.360543 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 393/806] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:13:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:26.360543 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25a18d2-14db-43ad-b963-ea90ef00aab4 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:26.421261 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daacc30-3117-42bf-9234-e2d7c6437c4b None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:26.474517 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9bfa93-5a33-409a-857b-777c07b4a2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:26.475745 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9bfa93-5a33-409a-857b-777c07b4a2b1 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:26.476291 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9bfa93-5a33-409a-857b-777c07b4a2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:26.476827 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9bfa93-5a33-409a-857b-777c07b4a2b1 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:26.478694 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9bfa93-5a33-409a-857b-777c07b4a2b1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:26.479320 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9bfa93-5a33-409a-857b-777c07b4a2b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:26.482298 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9bfa93-5a33-409a-857b-777c07b4a2b1 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:26.520009 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 414/807] 10.222.0.87 () {56 vars in 1149 bytes} [Tue Jun 21 18:13:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:26.521567 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9bfa93-5a33-409a-857b-777c07b4a2b1 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:26.537138 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25a18d2-14db-43ad-b963-ea90ef00aab4 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:26.577142 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:26.599011 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219d4281-0bf4-463d-8eb6-8c87ac7220a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:26.600282 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d4281-0bf4-463d-8eb6-8c87ac7220a5 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:26.600730 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d4281-0bf4-463d-8eb6-8c87ac7220a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:26.601176 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d4281-0bf4-463d-8eb6-8c87ac7220a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:26.617760 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219d4281-0bf4-463d-8eb6-8c87ac7220a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:26.619324 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219d4281-0bf4-463d-8eb6-8c87ac7220a5 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:26.640616 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-219d4281-0bf4-463d-8eb6-8c87ac7220a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:26.640616 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:26.640616 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:26.640616 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:26.640616 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:26.640616 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:26.640616 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:26.640616 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:26.640616 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:26.640616 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:26.640616 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:26.640616 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:26.640616 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:13:26.640616 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:13:26.640616 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:13:26.642498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:13:26.642498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:26.642498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:26.642498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:26.642498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:26.642498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:26.642498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:26.642498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:26.642498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:26.642498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:26.642498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:26.642498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:26.642498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:26.642498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:26.642498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:26.642498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:26.642498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:26.644334 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:13:26.644334 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:13:26.644334 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:26.644334 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:26.644334 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 394/808] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:13:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:26.644334 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d3d27-5781-4199-975b-61e8b2071e83 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:26.645115 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d3d27-5781-4199-975b-61e8b2071e83 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:26.645243 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d3d27-5781-4199-975b-61e8b2071e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:26.668904 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-9f1d3d27-5781-4199-975b-61e8b2071e83 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:13:26.670653 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9f1d3d27-5781-4199-975b-61e8b2071e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:13:26.692468 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:26.701743 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9bfa93-5a33-409a-857b-777c07b4a2b1 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:26.753621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec07508-5074-4e27-a3ed-0cdf34a8af53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:26.754573 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec07508-5074-4e27-a3ed-0cdf34a8af53 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:26.755020 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec07508-5074-4e27-a3ed-0cdf34a8af53 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:26.755475 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec07508-5074-4e27-a3ed-0cdf34a8af53 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:26.756743 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec07508-5074-4e27-a3ed-0cdf34a8af53 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:26.757307 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec07508-5074-4e27-a3ed-0cdf34a8af53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:26.760348 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec07508-5074-4e27-a3ed-0cdf34a8af53 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:26.774925 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 415/809] 10.222.0.87 () {56 vars in 1124 bytes} [Tue Jun 21 18:13:26 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:26.774925 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec07508-5074-4e27-a3ed-0cdf34a8af53 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:26.781131 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 395/810] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:13:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:13:26.782846 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1d3d27-5781-4199-975b-61e8b2071e83 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:26.950685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec07508-5074-4e27-a3ed-0cdf34a8af53 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:26.957994 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1d3d27-5781-4199-975b-61e8b2071e83 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:26.978090 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:27.021490 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40757461-c8e7-4bc6-9bb3-1577e97ac330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:27.022453 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40757461-c8e7-4bc6-9bb3-1577e97ac330 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:27.022879 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40757461-c8e7-4bc6-9bb3-1577e97ac330 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:27.023392 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40757461-c8e7-4bc6-9bb3-1577e97ac330 None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:27.024681 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40757461-c8e7-4bc6-9bb3-1577e97ac330 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:27.024913 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099865d-01e5-40b1-9b0f-90dfa0dfd950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:27.025149 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40757461-c8e7-4bc6-9bb3-1577e97ac330 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:27.026232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099865d-01e5-40b1-9b0f-90dfa0dfd950 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:27.026677 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099865d-01e5-40b1-9b0f-90dfa0dfd950 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:27.027112 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099865d-01e5-40b1-9b0f-90dfa0dfd950 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:27.027565 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40757461-c8e7-4bc6-9bb3-1577e97ac330 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:27.045741 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f099865d-01e5-40b1-9b0f-90dfa0dfd950 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:27.046066 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 416/811] 10.222.0.87 () {56 vars in 1148 bytes} [Tue Jun 21 18:13:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:27.046868 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40757461-c8e7-4bc6-9bb3-1577e97ac330 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:27.047442 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f099865d-01e5-40b1-9b0f-90dfa0dfd950 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:27.075712 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:27.077953 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-f099865d-01e5-40b1-9b0f-90dfa0dfd950 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:27.077953 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:27.077953 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:27.077953 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:27.077953 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:27.077953 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:27.077953 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:27.077953 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:27.077953 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:27.077953 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:27.077953 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:27.077953 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:27.077953 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:13:27.077953 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:13:27.077953 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:13:27.079168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:13:27.079168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:27.079168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:27.079168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:27.079168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:27.079168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:27.079168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:27.079168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:27.079168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:27.079168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:27.079168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:27.079168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:27.079168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:27.079168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:27.079168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:27.079168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:27.079168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:27.087046 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:13:27.087046 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:13:27.087046 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:27.087046 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:27.087046 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 396/812] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:13:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:27.087046 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099865d-01e5-40b1-9b0f-90dfa0dfd950 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:27.198599 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40757461-c8e7-4bc6-9bb3-1577e97ac330 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:27.241144 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099865d-01e5-40b1-9b0f-90dfa0dfd950 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:27.244952 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763edd8f-a000-4e54-ab0f-6bfb2a5108c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:27.245824 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763edd8f-a000-4e54-ab0f-6bfb2a5108c9 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:27.246268 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763edd8f-a000-4e54-ab0f-6bfb2a5108c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:27.246551 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763edd8f-a000-4e54-ab0f-6bfb2a5108c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:27.247366 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763edd8f-a000-4e54-ab0f-6bfb2a5108c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:27.247700 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763edd8f-a000-4e54-ab0f-6bfb2a5108c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:27.249121 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763edd8f-a000-4e54-ab0f-6bfb2a5108c9 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:27.263371 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 417/813] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:13:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:27.264260 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763edd8f-a000-4e54-ab0f-6bfb2a5108c9 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:27.281454 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d11af0-f80d-49aa-a30e-6371387c3ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:27.282143 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d11af0-f80d-49aa-a30e-6371387c3ea2 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:27.282442 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d11af0-f80d-49aa-a30e-6371387c3ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:27.282723 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d11af0-f80d-49aa-a30e-6371387c3ea2 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:27.283455 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d11af0-f80d-49aa-a30e-6371387c3ea2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:27.283778 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d11af0-f80d-49aa-a30e-6371387c3ea2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:27.285194 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d11af0-f80d-49aa-a30e-6371387c3ea2 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:27.291470 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 397/814] 10.222.0.87 () {56 vars in 1124 bytes} [Tue Jun 21 18:13:27 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:27.292756 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d11af0-f80d-49aa-a30e-6371387c3ea2 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:27.333443 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:27.375778 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:27.412636 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763edd8f-a000-4e54-ab0f-6bfb2a5108c9 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:27.467634 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d11af0-f80d-49aa-a30e-6371387c3ea2 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:27.467926 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04867f2f-56ba-440b-8a1e-dc54572f50e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:27.468803 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04867f2f-56ba-440b-8a1e-dc54572f50e2 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:27.469226 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04867f2f-56ba-440b-8a1e-dc54572f50e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:27.469699 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04867f2f-56ba-440b-8a1e-dc54572f50e2 None admin] PATH_INFO: `/v3/services/network` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:27.470849 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04867f2f-56ba-440b-8a1e-dc54572f50e2 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:27.473188 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04867f2f-56ba-440b-8a1e-dc54572f50e2 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:27.488692 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-04867f2f-56ba-440b-8a1e-dc54572f50e2 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 21 18:13:27.488692 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:27.488692 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:13:27.488692 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:13:27.488692 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:13:27.488692 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:13:27.488692 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:13:27.488692 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:13:27.488692 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 21 18:13:27.488692 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:27.488692 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:13:27.488692 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:27.488692 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:27.488692 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:27.488692 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:27.488692 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:27.488692 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:27.490796 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:27.490796 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:27.490796 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:27.490796 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:27.490796 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:27.490796 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:27.490796 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:13:27.490796 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:13:27.490796 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:13:27.490796 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:13:27.490796 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:27.490796 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:27.490796 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:27.490796 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:27.490796 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:27.490796 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:27.492957 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:27.492957 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:27.492957 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:27.492957 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:27.492957 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:27.492957 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:27.492957 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:27.492957 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:27.492957 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:27.492957 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:27.492957 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:13:27.492957 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:13:27.492957 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 21 18:13:27.492957 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:27.497822 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 418/815] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:13:27 2022] GET /identity/v3/services/network => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:27.499012 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04867f2f-56ba-440b-8a1e-dc54572f50e2 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:27.519454 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aacd877-977c-4a39-8535-281770c3878c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:27.520375 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aacd877-977c-4a39-8535-281770c3878c None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:27.520659 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aacd877-977c-4a39-8535-281770c3878c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:27.520942 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aacd877-977c-4a39-8535-281770c3878c None admin] PATH_INFO: `/v3/projects/5f30c3ddb5e44e9299ff228fe94c27fb/users/515b3571045e4fe29565307c8ecc10b2/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:27.523205 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aacd877-977c-4a39-8535-281770c3878c None admin] RBAC: Authorizing `identity:create_grant(project_id=5f30c3ddb5e44e9299ff228fe94c27fb, user_id=515b3571045e4fe29565307c8ecc10b2, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:27.524999 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aacd877-977c-4a39-8535-281770c3878c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:27.547281 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 398/816] 10.222.0.87 () {58 vars in 1439 bytes} [Tue Jun 21 18:13:27 2022] PUT /identity/v3/projects/5f30c3ddb5e44e9299ff228fe94c27fb/users/515b3571045e4fe29565307c8ecc10b2/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:13:27.548240 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aacd877-977c-4a39-8535-281770c3878c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:27.706798 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04867f2f-56ba-440b-8a1e-dc54572f50e2 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:27.732897 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:27.768810 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aacd877-977c-4a39-8535-281770c3878c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:27.774285 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedacbea-3a4e-4a5b-9dc3-b7b5ecdaf251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:27.774875 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedacbea-3a4e-4a5b-9dc3-b7b5ecdaf251 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:27.775147 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedacbea-3a4e-4a5b-9dc3-b7b5ecdaf251 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:27.775430 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedacbea-3a4e-4a5b-9dc3-b7b5ecdaf251 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:27.776929 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedacbea-3a4e-4a5b-9dc3-b7b5ecdaf251 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:27.776929 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedacbea-3a4e-4a5b-9dc3-b7b5ecdaf251 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:27.777930 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedacbea-3a4e-4a5b-9dc3-b7b5ecdaf251 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:27.795081 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 419/817] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:13:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:27.796565 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedacbea-3a4e-4a5b-9dc3-b7b5ecdaf251 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:27.807303 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:27.846069 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bac1db-f7a6-4e4b-8e80-4e61f52a0bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:27.847135 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bac1db-f7a6-4e4b-8e80-4e61f52a0bbe None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:27.847869 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bac1db-f7a6-4e4b-8e80-4e61f52a0bbe None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:27.848326 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bac1db-f7a6-4e4b-8e80-4e61f52a0bbe None admin] PATH_INFO: `/v3/users/nova` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:27.863028 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bac1db-f7a6-4e4b-8e80-4e61f52a0bbe None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:27.865506 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bac1db-f7a6-4e4b-8e80-4e61f52a0bbe None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:27.885869 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-b0bac1db-f7a6-4e4b-8e80-4e61f52a0bbe None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:13:27.885869 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:27.885869 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:27.885869 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:27.885869 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:27.885869 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:27.885869 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:27.885869 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:27.885869 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:27.885869 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:27.885869 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:27.885869 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:27.885869 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:13:27.885869 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:13:27.885869 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:13:27.885869 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:13:27.886889 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:27.886889 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:27.886889 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:13:27.886889 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:13:27.886889 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:13:27.886889 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:13:27.886889 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:27.886889 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:27.886889 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:27.886889 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:27.886889 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:27.886889 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:27.886889 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:27.886889 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:27.886889 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:27.886889 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:27.886889 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:27.886889 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:27.888300 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:27.888300 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:27.888300 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:13:27.888300 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:13:27.888300 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:13:27.888300 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:13:27.888300 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:13:27.888300 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:13:27.888300 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:13:27.888300 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:27.888300 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 399/818] 10.222.0.87 () {56 vars in 1093 bytes} [Tue Jun 21 18:13:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:27.889207 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bac1db-f7a6-4e4b-8e80-4e61f52a0bbe None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:28.009678 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedacbea-3a4e-4a5b-9dc3-b7b5ecdaf251 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:28.059143 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bac1db-f7a6-4e4b-8e80-4e61f52a0bbe None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:28.073840 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a0873-9a38-4beb-a9d8-382fcc87215a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:28.074923 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a0873-9a38-4beb-a9d8-382fcc87215a None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:28.075419 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a0873-9a38-4beb-a9d8-382fcc87215a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:28.075919 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a0873-9a38-4beb-a9d8-382fcc87215a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:28.077786 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2a0873-9a38-4beb-a9d8-382fcc87215a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:28.079239 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2a0873-9a38-4beb-a9d8-382fcc87215a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:28.114121 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 420/819] 10.222.0.87 () {60 vars in 1144 bytes} [Tue Jun 21 18:13:27 2022] POST /identity/v3/endpoints => generated 346 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:13:28.115105 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a0873-9a38-4beb-a9d8-382fcc87215a None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:28.159726 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b539af7-e3b2-499c-a30e-705ec11d992e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:28.160118 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b539af7-e3b2-499c-a30e-705ec11d992e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:28.160742 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b539af7-e3b2-499c-a30e-705ec11d992e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:28.161296 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b539af7-e3b2-499c-a30e-705ec11d992e None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:28.162545 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b539af7-e3b2-499c-a30e-705ec11d992e None admin] RBAC: Adding query filter params (name=network) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:28.162907 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b539af7-e3b2-499c-a30e-705ec11d992e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:28.165057 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b539af7-e3b2-499c-a30e-705ec11d992e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:28.178595 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 400/820] 10.222.0.87 () {56 vars in 1112 bytes} [Tue Jun 21 18:13:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:28.180526 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b539af7-e3b2-499c-a30e-705ec11d992e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:28.186941 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:28.237190 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:28.331117 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a0873-9a38-4beb-a9d8-382fcc87215a None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:28.348505 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b539af7-e3b2-499c-a30e-705ec11d992e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:28.380166 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335b901a-7286-4d23-bc12-8f5ec4849789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:28.381368 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b901a-7286-4d23-bc12-8f5ec4849789 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:28.381960 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b901a-7286-4d23-bc12-8f5ec4849789 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:28.382437 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b901a-7286-4d23-bc12-8f5ec4849789 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:28.393126 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335b901a-7286-4d23-bc12-8f5ec4849789 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:28.393401 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335b901a-7286-4d23-bc12-8f5ec4849789 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:28.402187 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32618305-1cf5-4c60-aed7-5e1134495dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:28.402669 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32618305-1cf5-4c60-aed7-5e1134495dcd None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:28.402989 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32618305-1cf5-4c60-aed7-5e1134495dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:28.403264 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32618305-1cf5-4c60-aed7-5e1134495dcd None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:28.404135 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32618305-1cf5-4c60-aed7-5e1134495dcd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:28.404446 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32618305-1cf5-4c60-aed7-5e1134495dcd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:28.407229 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32618305-1cf5-4c60-aed7-5e1134495dcd None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:28.411846 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-335b901a-7286-4d23-bc12-8f5ec4849789 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:13:28.411846 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:28.411846 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:28.411846 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:28.411846 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:28.411846 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:28.411846 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:28.411846 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:28.411846 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:28.411846 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:28.411846 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:28.411846 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:28.411846 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:13:28.411846 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:13:28.411846 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:13:28.413923 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:13:28.413923 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:28.413923 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:28.413923 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:28.413923 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:28.413923 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:28.413923 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:28.413923 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:28.413923 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:28.413923 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:28.413923 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:28.413923 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:28.413923 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:28.413923 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:28.413923 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:28.413923 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:28.413923 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:28.416198 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:13:28.416198 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:13:28.416198 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:13:28.416198 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:13:28.416198 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:13:28.416198 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:13:28.416198 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:13:28.416198 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:28.419839 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 421/821] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:13:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:28.419839 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335b901a-7286-4d23-bc12-8f5ec4849789 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:28.432802 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 401/822] 10.222.0.87 () {56 vars in 1133 bytes} [Tue Jun 21 18:13:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:28.432802 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32618305-1cf5-4c60-aed7-5e1134495dcd None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:28.569446 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:28.593473 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32618305-1cf5-4c60-aed7-5e1134495dcd None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:28.627775 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335b901a-7286-4d23-bc12-8f5ec4849789 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:28.657086 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa2b567-c076-4294-9e84-01041debb9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:28.658543 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa2b567-c076-4294-9e84-01041debb9c2 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:28.658990 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa2b567-c076-4294-9e84-01041debb9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:28.659426 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa2b567-c076-4294-9e84-01041debb9c2 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:28.661050 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa2b567-c076-4294-9e84-01041debb9c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:28.661549 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa2b567-c076-4294-9e84-01041debb9c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:28.662277 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:28.665214 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa2b567-c076-4294-9e84-01041debb9c2 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:28.692689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 402/823] 10.222.0.87 () {56 vars in 1148 bytes} [Tue Jun 21 18:13:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:28.694104 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa2b567-c076-4294-9e84-01041debb9c2 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:28.696741 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a94f4-e592-4924-8507-fea422e58154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:28.697986 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a94f4-e592-4924-8507-fea422e58154 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:28.698528 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a94f4-e592-4924-8507-fea422e58154 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:28.699126 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a94f4-e592-4924-8507-fea422e58154 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:28.700472 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790a94f4-e592-4924-8507-fea422e58154 None admin] RBAC: Adding query filter params (type=network) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:28.701053 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790a94f4-e592-4924-8507-fea422e58154 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:28.703577 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790a94f4-e592-4924-8507-fea422e58154 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:28.721793 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 422/824] 10.222.0.87 () {56 vars in 1112 bytes} [Tue Jun 21 18:13:28 2022] GET /identity/v3/services?type=network => generated 350 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:28.723716 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a94f4-e592-4924-8507-fea422e58154 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:28.901397 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a94f4-e592-4924-8507-fea422e58154 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:28.901732 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa2b567-c076-4294-9e84-01041debb9c2 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:28.952573 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:28.967278 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95801a01-4735-475a-9f91-e4277869b594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:28.968948 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95801a01-4735-475a-9f91-e4277869b594 None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:28.969355 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95801a01-4735-475a-9f91-e4277869b594 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:28.974973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95801a01-4735-475a-9f91-e4277869b594 None admin] PATH_INFO: `/v3/projects/5f30c3ddb5e44e9299ff228fe94c27fb/users/09c9fa7dccb8491c9ad3b43967345ea2/roles/b9dbf55e6f2947158c1d946f6ee9f907` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:28.976730 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1669b5-541c-46a5-a8eb-4f5df7768e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:28.978647 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1669b5-541c-46a5-a8eb-4f5df7768e0e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:28.979148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1669b5-541c-46a5-a8eb-4f5df7768e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:28.979641 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1669b5-541c-46a5-a8eb-4f5df7768e0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:29.006422 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95801a01-4735-475a-9f91-e4277869b594 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f30c3ddb5e44e9299ff228fe94c27fb, user_id=09c9fa7dccb8491c9ad3b43967345ea2, role_id=b9dbf55e6f2947158c1d946f6ee9f907)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:29.008663 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95801a01-4735-475a-9f91-e4277869b594 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:29.027247 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 423/825] 10.222.0.87 () {58 vars in 1439 bytes} [Tue Jun 21 18:13:28 2022] PUT /identity/v3/projects/5f30c3ddb5e44e9299ff228fe94c27fb/users/09c9fa7dccb8491c9ad3b43967345ea2/roles/b9dbf55e6f2947158c1d946f6ee9f907 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:13:29.028249 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95801a01-4735-475a-9f91-e4277869b594 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:29.081888 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:29.134801 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1669b5-541c-46a5-a8eb-4f5df7768e0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:29.138082 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1669b5-541c-46a5-a8eb-4f5df7768e0e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:29.239890 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95801a01-4735-475a-9f91-e4277869b594 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:29.313701 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531a451d-8d90-49a9-95ba-86d7794950de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:29.317318 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a451d-8d90-49a9-95ba-86d7794950de None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:29.318228 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a451d-8d90-49a9-95ba-86d7794950de None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:29.322445 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a451d-8d90-49a9-95ba-86d7794950de None admin] PATH_INFO: `/v3/endpoints` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:29.324446 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531a451d-8d90-49a9-95ba-86d7794950de None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:29.330942 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531a451d-8d90-49a9-95ba-86d7794950de None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:29.388049 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 403/826] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:13:28 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 691 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:13:29.391165 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1669b5-541c-46a5-a8eb-4f5df7768e0e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:29.403562 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 424/827] 10.222.0.87 () {60 vars in 1144 bytes} [Tue Jun 21 18:13:29 2022] POST /identity/v3/endpoints => generated 327 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:13:29.468773 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:29.658486 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1669b5-541c-46a5-a8eb-4f5df7768e0e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:29.763034 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a516a0-ec54-4b4c-b005-31faa753ec04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:29.764482 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a516a0-ec54-4b4c-b005-31faa753ec04 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:29.765004 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a516a0-ec54-4b4c-b005-31faa753ec04 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:29.765511 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a516a0-ec54-4b4c-b005-31faa753ec04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:29.794257 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a516a0-ec54-4b4c-b005-31faa753ec04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:29.797148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a516a0-ec54-4b4c-b005-31faa753ec04 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:29.842366 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-64a516a0-ec54-4b4c-b005-31faa753ec04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:29.842366 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:29.842366 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:29.842366 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:29.842366 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:29.842366 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:29.842366 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:29.842366 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:29.842366 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:29.842366 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:29.842366 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:29.842366 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:29.842366 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:13:29.842366 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:13:29.842366 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:13:29.844224 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:13:29.844224 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:29.844224 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:29.844224 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:29.844224 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:29.844224 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:29.844224 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:29.844224 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:29.844224 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:29.844224 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:29.844224 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:29.844224 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:29.844224 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:29.844224 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:29.844224 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:29.844224 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:29.844224 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:29.846591 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:13:29.846591 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:13:29.846591 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:29.846591 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:29.846591 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 404/828] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:13:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 454 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:29.859442 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531a451d-8d90-49a9-95ba-86d7794950de None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:30.068124 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:30.101524 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531a451d-8d90-49a9-95ba-86d7794950de None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:30.185975 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed7a1f-6053-4ed7-9336-c24f4aade84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:30.187112 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed7a1f-6053-4ed7-9336-c24f4aade84e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:30.187583 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed7a1f-6053-4ed7-9336-c24f4aade84e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:30.188053 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed7a1f-6053-4ed7-9336-c24f4aade84e None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:30.189414 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ed7a1f-6053-4ed7-9336-c24f4aade84e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:30.189983 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ed7a1f-6053-4ed7-9336-c24f4aade84e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:30.192526 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ed7a1f-6053-4ed7-9336-c24f4aade84e None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:30.221939 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 425/829] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:13:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:30.231361 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a516a0-ec54-4b4c-b005-31faa753ec04 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:30.330116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:30.382607 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a516a0-ec54-4b4c-b005-31faa753ec04 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:30.449331 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ea169e-e8bf-4410-a22a-6ec7159d2796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:30.450695 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ea169e-e8bf-4410-a22a-6ec7159d2796 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:30.451221 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ea169e-e8bf-4410-a22a-6ec7159d2796 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:30.451729 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ea169e-e8bf-4410-a22a-6ec7159d2796 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:30.463478 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ea169e-e8bf-4410-a22a-6ec7159d2796 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:30.466740 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ea169e-e8bf-4410-a22a-6ec7159d2796 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:30.493176 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-f0ea169e-e8bf-4410-a22a-6ec7159d2796 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:13:30.493176 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:30.493176 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:30.493176 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:30.493176 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:30.493176 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:30.493176 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:30.493176 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:30.493176 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:30.493176 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:30.493176 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:30.493176 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:30.493176 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:13:30.493176 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:13:30.493176 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:13:30.494211 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:13:30.494211 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:30.494211 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:30.494211 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:30.494211 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:30.494211 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:30.494211 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:30.494211 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:30.494211 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:30.494211 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:30.494211 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:30.494211 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:30.494211 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:30.494211 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:30.494211 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:30.494211 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:30.494211 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:30.495112 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:13:30.495112 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:13:30.495112 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:13:30.495112 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:13:30.495112 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:13:30.495112 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:13:30.495112 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:13:30.495112 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:30.501827 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 405/830] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:13:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:30.506864 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed7a1f-6053-4ed7-9336-c24f4aade84e None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:30.699952 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed7a1f-6053-4ed7-9336-c24f4aade84e None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:30.760394 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32faa80-8e61-4413-a04c-c3dd2c3a38c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:30.761082 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32faa80-8e61-4413-a04c-c3dd2c3a38c7 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:30.761366 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32faa80-8e61-4413-a04c-c3dd2c3a38c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:30.761666 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32faa80-8e61-4413-a04c-c3dd2c3a38c7 None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:30.762485 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32faa80-8e61-4413-a04c-c3dd2c3a38c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:30.762946 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32faa80-8e61-4413-a04c-c3dd2c3a38c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:30.764717 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32faa80-8e61-4413-a04c-c3dd2c3a38c7 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:30.779119 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 426/831] 10.222.0.87 () {56 vars in 1148 bytes} [Tue Jun 21 18:13:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:30.786138 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ea169e-e8bf-4410-a22a-6ec7159d2796 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:30.933197 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ea169e-e8bf-4410-a22a-6ec7159d2796 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:30.985259 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:30.995409 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64234b16-f18f-49c7-947f-de45744489d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:30.996601 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64234b16-f18f-49c7-947f-de45744489d3 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:30.997108 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64234b16-f18f-49c7-947f-de45744489d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:30.997790 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64234b16-f18f-49c7-947f-de45744489d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:30.998983 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64234b16-f18f-49c7-947f-de45744489d3 None admin] RBAC: Adding query filter params (role.id=98f0943a610f4a9e9f8de54f6e9f717d, scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb, user.id=a1b3fa67905c4027b3656ac36e2b76c9) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:30.999575 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64234b16-f18f-49c7-947f-de45744489d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:31.001318 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64234b16-f18f-49c7-947f-de45744489d3 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:31.015352 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 406/832] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:13:30 2022] GET /identity/v3/role_assignments?user.id=a1b3fa67905c4027b3656ac36e2b76c9&scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb&role.id=98f0943a610f4a9e9f8de54f6e9f717d => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:31.173728 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991a00b4-f08c-4c70-851a-3040385087e1 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:31.174547 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991a00b4-f08c-4c70-851a-3040385087e1 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:31.174547 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991a00b4-f08c-4c70-851a-3040385087e1 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:31.176092 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 427/833] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:13:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:13:31.183786 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb304dc-7f3f-4be0-aed0-7f0c4c013760 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:31.184145 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb304dc-7f3f-4be0-aed0-7f0c4c013760 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:31.184363 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb304dc-7f3f-4be0-aed0-7f0c4c013760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:31.206970 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-4bb304dc-7f3f-4be0-aed0-7f0c4c013760 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:13:31.209081 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4bb304dc-7f3f-4be0-aed0-7f0c4c013760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:13:31.370862 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 407/834] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:13:31 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:13:31.396147 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991a00b4-f08c-4c70-851a-3040385087e1 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:31.411387 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:31.421121 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da56c0-8db5-4b07-9372-2e3badc40465 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:31.422119 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da56c0-8db5-4b07-9372-2e3badc40465 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:31.423032 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da56c0-8db5-4b07-9372-2e3badc40465 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:31.428369 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 408/835] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:13:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:13:31.436650 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ddbb64-d7b8-4e4f-bed9-a22893568864 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:31.437564 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ddbb64-d7b8-4e4f-bed9-a22893568864 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:31.438112 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ddbb64-d7b8-4e4f-bed9-a22893568864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:31.472878 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a3ddbb64-d7b8-4e4f-bed9-a22893568864 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:13:31.475155 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a3ddbb64-d7b8-4e4f-bed9-a22893568864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:13:31.594448 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991a00b4-f08c-4c70-851a-3040385087e1 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:31.613874 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 409/836] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:13:31 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:13:31.644819 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ddbb64-d7b8-4e4f-bed9-a22893568864 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:31.670222 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e58efe6-43f0-47b7-b1fc-4cf5c9d0ff60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:31.672287 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58efe6-43f0-47b7-b1fc-4cf5c9d0ff60 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:31.673094 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58efe6-43f0-47b7-b1fc-4cf5c9d0ff60 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:31.673925 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58efe6-43f0-47b7-b1fc-4cf5c9d0ff60 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:31.698317 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e58efe6-43f0-47b7-b1fc-4cf5c9d0ff60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:13:31.715419 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-2e58efe6-43f0-47b7-b1fc-4cf5c9d0ff60 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:13:31.715419 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:31.715419 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:31.715419 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:31.715419 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:31.715419 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:31.715419 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:31.715419 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:31.715419 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:31.715419 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:31.715419 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:31.715419 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:31.715419 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:13:31.715419 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:13:31.715419 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:13:31.715419 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:13:31.717146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:13:31.717146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:13:31.717146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:13:31.717146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:13:31.717146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:31.717146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:31.717146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:31.717146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:31.717146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:31.717146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:31.717146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:31.717146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:31.717146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:31.717146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:31.717146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:31.717146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:31.718591 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:31.718591 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:31.718591 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:31.718591 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:31.718591 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:13:31.718591 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:13:31.718591 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:13:31.718591 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:13:31.718591 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:13:31.718591 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:13:31.718591 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:13:31.718591 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:31.730139 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 428/837] 10.222.0.87 () {56 vars in 1096 bytes} [Tue Jun 21 18:13:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:31.735156 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e58efe6-43f0-47b7-b1fc-4cf5c9d0ff60 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:31.758288 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:31.859671 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ddbb64-d7b8-4e4f-bed9-a22893568864 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:31.942802 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:31.952664 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd9daa4-47f1-49e3-9f1a-65b822d78919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:31.953860 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9daa4-47f1-49e3-9f1a-65b822d78919 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:31.954115 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9daa4-47f1-49e3-9f1a-65b822d78919 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:31.954415 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9daa4-47f1-49e3-9f1a-65b822d78919 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:31.955589 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd9daa4-47f1-49e3-9f1a-65b822d78919 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:31.958390 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd9daa4-47f1-49e3-9f1a-65b822d78919 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:31.969925 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e58efe6-43f0-47b7-b1fc-4cf5c9d0ff60 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:31.988771 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-0dd9daa4-47f1-49e3-9f1a-65b822d78919 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 21 18:13:31.988771 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:31.988771 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:13:31.988771 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:13:31.988771 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:13:31.988771 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:13:31.988771 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:13:31.988771 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:13:31.988771 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 21 18:13:31.988771 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:31.988771 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:13:31.988771 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:31.988771 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:31.988771 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:31.988771 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:31.988771 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:31.988771 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:31.990801 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:31.990801 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:31.990801 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:31.990801 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:31.990801 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:31.990801 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:31.990801 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:13:31.990801 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:13:31.990801 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:13:31.990801 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:13:31.990801 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:31.990801 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:31.990801 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:31.990801 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:31.990801 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:31.990801 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:31.992217 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:31.992217 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:31.992217 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:31.992217 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:31.992217 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:31.992217 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:31.992217 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:31.992217 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:31.992217 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:31.992217 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:31.992217 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:13:31.992217 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:13:31.992217 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 21 18:13:31.992217 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:31.998585 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 410/838] 10.222.0.87 () {56 vars in 1114 bytes} [Tue Jun 21 18:13:31 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:13:32.000764 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd9daa4-47f1-49e3-9f1a-65b822d78919 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:32.055141 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd794b3f-274f-4b2d-8a22-777802450d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:32.056416 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd794b3f-274f-4b2d-8a22-777802450d5d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:32.056954 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd794b3f-274f-4b2d-8a22-777802450d5d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:32.057665 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd794b3f-274f-4b2d-8a22-777802450d5d None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:32.059727 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd794b3f-274f-4b2d-8a22-777802450d5d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:32.059935 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd794b3f-274f-4b2d-8a22-777802450d5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:32.062681 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd794b3f-274f-4b2d-8a22-777802450d5d None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:32.083769 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 429/839] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:13:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:13:32.092231 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd794b3f-274f-4b2d-8a22-777802450d5d None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:32.213267 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd9daa4-47f1-49e3-9f1a-65b822d78919 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:32.252501 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:32.285789 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27711bd-96c5-429d-84a7-d1598f3f17fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:32.287359 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27711bd-96c5-429d-84a7-d1598f3f17fc None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:32.288003 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27711bd-96c5-429d-84a7-d1598f3f17fc None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:32.288668 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27711bd-96c5-429d-84a7-d1598f3f17fc None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:32.290629 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27711bd-96c5-429d-84a7-d1598f3f17fc None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:32.291254 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27711bd-96c5-429d-84a7-d1598f3f17fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:32.294250 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27711bd-96c5-429d-84a7-d1598f3f17fc None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:32.303339 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 411/840] 10.222.0.87 () {56 vars in 1114 bytes} [Tue Jun 21 18:13:31 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:13:32.312028 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd794b3f-274f-4b2d-8a22-777802450d5d None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:32.312652 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27711bd-96c5-429d-84a7-d1598f3f17fc None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:32.374340 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:32.390216 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade2cbfd-da64-4467-b11a-bc8fa0a45810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:32.391939 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2cbfd-da64-4467-b11a-bc8fa0a45810 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:32.392718 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2cbfd-da64-4467-b11a-bc8fa0a45810 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:32.393986 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2cbfd-da64-4467-b11a-bc8fa0a45810 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:32.425745 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade2cbfd-da64-4467-b11a-bc8fa0a45810 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:32.431715 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade2cbfd-da64-4467-b11a-bc8fa0a45810 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:32.472608 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-ade2cbfd-da64-4467-b11a-bc8fa0a45810 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:32.472608 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:32.472608 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:32.472608 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:32.472608 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:32.472608 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:32.472608 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:32.472608 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:32.472608 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:32.472608 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:32.472608 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:32.472608 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:32.472608 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:13:32.472608 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:13:32.472608 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:13:32.474598 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:13:32.474598 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:32.474598 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:32.474598 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:32.474598 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:32.474598 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:32.474598 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:32.474598 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:32.474598 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:32.474598 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:32.474598 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:32.474598 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:32.474598 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:32.474598 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:32.474598 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:32.474598 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:32.474598 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:32.476521 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:13:32.476521 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:13:32.476521 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:32.476521 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:32.482502 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 430/841] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:13:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 391 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:32.486604 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade2cbfd-da64-4467-b11a-bc8fa0a45810 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:32.573018 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27711bd-96c5-429d-84a7-d1598f3f17fc None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:32.653610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b555be5-afe7-438c-8aba-9f5d3fdd59d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:32.655689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b555be5-afe7-438c-8aba-9f5d3fdd59d8 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:32.656557 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b555be5-afe7-438c-8aba-9f5d3fdd59d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:32.657390 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b555be5-afe7-438c-8aba-9f5d3fdd59d8 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:32.659557 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b555be5-afe7-438c-8aba-9f5d3fdd59d8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:32.660619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b555be5-afe7-438c-8aba-9f5d3fdd59d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:32.663964 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b555be5-afe7-438c-8aba-9f5d3fdd59d8 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:32.681931 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 412/842] 10.222.0.87 () {56 vars in 1114 bytes} [Tue Jun 21 18:13:32 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:32.882240 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c2feaa-cbc8-416d-bd48-38130e4bdf35 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:32.882746 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c2feaa-cbc8-416d-bd48-38130e4bdf35 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:32.884028 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c2feaa-cbc8-416d-bd48-38130e4bdf35 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:32.890185 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 413/843] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:13:32 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:13:32.895886 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bc84c-30e0-4c07-9f93-d044ff9aa01a None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:32.896391 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bc84c-30e0-4c07-9f93-d044ff9aa01a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:32.896882 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bc84c-30e0-4c07-9f93-d044ff9aa01a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:32.950781 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f48bc84c-30e0-4c07-9f93-d044ff9aa01a None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:13:32.953771 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f48bc84c-30e0-4c07-9f93-d044ff9aa01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:13:32.975048 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade2cbfd-da64-4467-b11a-bc8fa0a45810 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:33.055205 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2b0e2-d522-4f2e-a102-ff14a225973c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:33.057947 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2b0e2-d522-4f2e-a102-ff14a225973c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:33.058499 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2b0e2-d522-4f2e-a102-ff14a225973c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:33.059003 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2b0e2-d522-4f2e-a102-ff14a225973c None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:33.060485 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f2b0e2-d522-4f2e-a102-ff14a225973c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:33.061044 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f2b0e2-d522-4f2e-a102-ff14a225973c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:33.063816 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f2b0e2-d522-4f2e-a102-ff14a225973c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:33.074068 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:33.090523 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 431/844] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:13:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 606 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:13:33.098933 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2b0e2-d522-4f2e-a102-ff14a225973c None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:33.126861 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 414/845] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:13:32 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:13:33.171798 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:33.173258 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48bc84c-30e0-4c07-9f93-d044ff9aa01a None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:33.285103 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2b0e2-d522-4f2e-a102-ff14a225973c None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:33.355406 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2fd40c-79e2-492c-9f2a-8a4d7233aa1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:33.356359 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2fd40c-79e2-492c-9f2a-8a4d7233aa1e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:33.356737 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2fd40c-79e2-492c-9f2a-8a4d7233aa1e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:33.357194 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2fd40c-79e2-492c-9f2a-8a4d7233aa1e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:33.372917 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2fd40c-79e2-492c-9f2a-8a4d7233aa1e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:33.375481 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2fd40c-79e2-492c-9f2a-8a4d7233aa1e None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:33.387072 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48bc84c-30e0-4c07-9f93-d044ff9aa01a None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:33.403729 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-7d2fd40c-79e2-492c-9f2a-8a4d7233aa1e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 21 18:13:33.403729 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:33.403729 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:33.403729 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:33.403729 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:33.403729 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:33.403729 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:33.403729 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:33.403729 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:33.403729 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:33.403729 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:33.403729 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:33.403729 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:13:33.403729 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:13:33.403729 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:13:33.403729 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:13:33.405550 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:33.405550 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:33.405550 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:13:33.405550 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:13:33.405550 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:13:33.405550 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:13:33.405550 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:33.405550 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:33.405550 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:33.405550 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:33.405550 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:33.405550 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:33.405550 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:33.405550 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:33.405550 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:33.405550 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:33.405550 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:33.405550 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:33.407393 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:33.407393 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:33.407393 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:13:33.407393 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:13:33.407393 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:13:33.407393 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:13:33.407393 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:13:33.407393 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:13:33.407393 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 21 18:13:33.407393 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:33.413801 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 432/846] 10.222.0.87 () {56 vars in 1096 bytes} [Tue Jun 21 18:13:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:33.415823 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2fd40c-79e2-492c-9f2a-8a4d7233aa1e None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:33.454474 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386db702-4aa1-49f1-8f8e-b61996a5c511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:33.456122 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386db702-4aa1-49f1-8f8e-b61996a5c511 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:33.456652 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386db702-4aa1-49f1-8f8e-b61996a5c511 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:33.457151 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386db702-4aa1-49f1-8f8e-b61996a5c511 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:33.473052 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386db702-4aa1-49f1-8f8e-b61996a5c511 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:13:33.482913 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-386db702-4aa1-49f1-8f8e-b61996a5c511 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:13:33.482913 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:33.482913 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:33.482913 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:33.482913 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:33.482913 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:33.482913 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:33.482913 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:33.482913 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:33.482913 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:33.482913 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:33.482913 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:33.482913 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:13:33.482913 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:13:33.482913 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:13:33.482913 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:13:33.483921 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:13:33.483921 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:13:33.483921 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:13:33.483921 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:13:33.483921 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:33.483921 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:33.483921 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:33.483921 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:33.483921 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:33.483921 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:33.483921 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:33.483921 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:33.483921 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:33.483921 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:33.483921 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:33.483921 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:33.484739 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:33.484739 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:33.484739 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:33.484739 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:33.484739 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:13:33.484739 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:13:33.484739 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:13:33.484739 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:13:33.484739 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:13:33.484739 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:13:33.484739 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:13:33.484739 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:33.493094 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 415/847] 10.222.0.87 () {56 vars in 1102 bytes} [Tue Jun 21 18:13:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:13:33.494560 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386db702-4aa1-49f1-8f8e-b61996a5c511 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:33.545964 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:33.628575 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2fd40c-79e2-492c-9f2a-8a4d7233aa1e None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:33.632382 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:33.672354 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386db702-4aa1-49f1-8f8e-b61996a5c511 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:33.701331 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d8e04d-ffc5-4509-a53d-bdf87d8a9b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:33.702567 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d8e04d-ffc5-4509-a53d-bdf87d8a9b35 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:33.703059 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d8e04d-ffc5-4509-a53d-bdf87d8a9b35 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:33.703548 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d8e04d-ffc5-4509-a53d-bdf87d8a9b35 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:33.705796 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d8e04d-ffc5-4509-a53d-bdf87d8a9b35 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:33.705796 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d8e04d-ffc5-4509-a53d-bdf87d8a9b35 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:33.708485 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d8e04d-ffc5-4509-a53d-bdf87d8a9b35 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:33.724847 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01a4eb1-63f3-471c-b44a-a71191e98e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:33.727010 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a4eb1-63f3-471c-b44a-a71191e98e9b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:33.727610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a4eb1-63f3-471c-b44a-a71191e98e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:33.728103 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a4eb1-63f3-471c-b44a-a71191e98e9b None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:33.729225 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01a4eb1-63f3-471c-b44a-a71191e98e9b None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:33.729811 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01a4eb1-63f3-471c-b44a-a71191e98e9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:33.731661 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01a4eb1-63f3-471c-b44a-a71191e98e9b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:33.748657 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 416/849] 10.222.0.87 () {56 vars in 1103 bytes} [Tue Jun 21 18:13:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:33.750098 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01a4eb1-63f3-471c-b44a-a71191e98e9b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:33.754034 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 433/849] 10.222.0.87 () {56 vars in 1135 bytes} [Tue Jun 21 18:13:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:33.756185 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d8e04d-ffc5-4509-a53d-bdf87d8a9b35 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:33.915353 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:33.923461 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01a4eb1-63f3-471c-b44a-a71191e98e9b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:33.969196 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d8e04d-ffc5-4509-a53d-bdf87d8a9b35 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:34.017201 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f846c7-8816-465c-8c4d-e040ce4185b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:34.018695 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f846c7-8816-465c-8c4d-e040ce4185b2 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:34.019186 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f846c7-8816-465c-8c4d-e040ce4185b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:34.019674 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f846c7-8816-465c-8c4d-e040ce4185b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:34.046910 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f846c7-8816-465c-8c4d-e040ce4185b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:34.051018 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f846c7-8816-465c-8c4d-e040ce4185b2 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:34.054480 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d901cd-495a-4109-87f0-2dff109a783a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:34.056059 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d901cd-495a-4109-87f0-2dff109a783a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:34.057230 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d901cd-495a-4109-87f0-2dff109a783a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:34.057917 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d901cd-495a-4109-87f0-2dff109a783a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:34.096168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-13f846c7-8816-465c-8c4d-e040ce4185b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:34.096168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:34.096168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:34.096168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:34.096168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:34.096168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:34.096168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:34.096168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:34.096168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:34.096168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:34.096168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:34.096168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:34.096168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:13:34.096168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:13:34.096168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:13:34.097751 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:13:34.097751 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:34.097751 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:34.097751 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:34.097751 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:34.097751 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:34.097751 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:34.097751 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:34.097751 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:34.097751 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:34.097751 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:34.097751 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:34.097751 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:34.097751 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:34.097751 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:34.097751 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:34.097751 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:34.098974 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:13:34.098974 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:13:34.098974 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:34.098974 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:34.101837 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 417/850] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:13:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:34.108904 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f846c7-8816-465c-8c4d-e040ce4185b2 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:34.114241 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:34.201827 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d901cd-495a-4109-87f0-2dff109a783a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:34.204961 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d901cd-495a-4109-87f0-2dff109a783a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:34.267040 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f846c7-8816-465c-8c4d-e040ce4185b2 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:34.318564 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac09a25f-1f9e-4d74-b286-2c74165b9e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:34.319933 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac09a25f-1f9e-4d74-b286-2c74165b9e96 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:34.320549 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac09a25f-1f9e-4d74-b286-2c74165b9e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:34.320979 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac09a25f-1f9e-4d74-b286-2c74165b9e96 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:34.321940 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac09a25f-1f9e-4d74-b286-2c74165b9e96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:34.322469 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac09a25f-1f9e-4d74-b286-2c74165b9e96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:34.324125 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac09a25f-1f9e-4d74-b286-2c74165b9e96 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:34.344894 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 418/851] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:13:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:34.347377 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9824fa9-13ee-4cee-89d9-7d8e9f6eba22 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:34.347821 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9824fa9-13ee-4cee-89d9-7d8e9f6eba22 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:34.348261 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9824fa9-13ee-4cee-89d9-7d8e9f6eba22 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:34.351330 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 419/852] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:13:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:13:34.356631 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9824fa9-13ee-4cee-89d9-7d8e9f6eba22 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:34.375863 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:34.419899 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 434/853] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:13:33 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 665 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:13:34.422290 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c701b9da-a85a-40d2-bad5-f63d82300c60 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:34.422804 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c701b9da-a85a-40d2-bad5-f63d82300c60 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:34.423306 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c701b9da-a85a-40d2-bad5-f63d82300c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:34.453203 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-c701b9da-a85a-40d2-bad5-f63d82300c60 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:13:34.455281 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c701b9da-a85a-40d2-bad5-f63d82300c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:13:34.504065 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9824fa9-13ee-4cee-89d9-7d8e9f6eba22 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:34.571637 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38010b86-407a-49eb-8bd0-857ebb439974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:34.573111 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38010b86-407a-49eb-8bd0-857ebb439974 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:34.573619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38010b86-407a-49eb-8bd0-857ebb439974 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:34.574197 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38010b86-407a-49eb-8bd0-857ebb439974 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:34.578399 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:34.591559 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38010b86-407a-49eb-8bd0-857ebb439974 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:34.594091 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38010b86-407a-49eb-8bd0-857ebb439974 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:34.605840 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 435/854] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:13:34 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:13:34.605840 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c701b9da-a85a-40d2-bad5-f63d82300c60 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:34.632751 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-38010b86-407a-49eb-8bd0-857ebb439974 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 18:13:34.632751 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:34.632751 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:34.632751 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:34.632751 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:34.632751 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:34.632751 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:34.632751 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:34.632751 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:34.632751 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:34.632751 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:34.632751 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:34.632751 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:13:34.632751 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:13:34.632751 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:13:34.632751 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:13:34.634717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:34.634717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:34.634717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:13:34.634717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:13:34.634717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:13:34.634717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:13:34.634717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:34.634717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:34.634717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:34.634717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:34.634717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:34.634717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:34.634717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:34.634717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:34.634717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:34.634717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:34.634717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:34.634717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:34.636772 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:34.636772 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:34.636772 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:13:34.636772 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:13:34.636772 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:13:34.636772 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:13:34.636772 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:13:34.636772 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:13:34.636772 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 18:13:34.636772 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:34.640369 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 420/855] 10.222.0.87 () {56 vars in 1117 bytes} [Tue Jun 21 18:13:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:34.641913 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38010b86-407a-49eb-8bd0-857ebb439974 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:34.747372 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:34.809087 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c701b9da-a85a-40d2-bad5-f63d82300c60 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:34.811134 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38010b86-407a-49eb-8bd0-857ebb439974 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:34.864413 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9fb3a0-a3cc-4404-9174-adcbd635295d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:34.865352 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9fb3a0-a3cc-4404-9174-adcbd635295d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:34.865761 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9fb3a0-a3cc-4404-9174-adcbd635295d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:34.866163 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9fb3a0-a3cc-4404-9174-adcbd635295d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:34.880613 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9fb3a0-a3cc-4404-9174-adcbd635295d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:34.882866 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9fb3a0-a3cc-4404-9174-adcbd635295d None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:34.902935 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42864f92-f6d7-4ded-bea4-69648b174cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:34.904581 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-9c9fb3a0-a3cc-4404-9174-adcbd635295d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:34.904581 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:34.904581 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:34.904581 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:34.904581 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:34.904581 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:34.904581 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:34.904581 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:34.904581 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:34.904581 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:34.904581 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:34.904581 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:34.904581 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:13:34.904581 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:13:34.904581 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:13:34.906416 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:13:34.906416 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:34.906416 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:34.906416 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:34.906416 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:34.906416 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:34.906416 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:34.906416 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:34.906416 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:34.906416 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:34.906416 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:34.906416 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:34.906416 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:34.906416 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:34.906416 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:34.906416 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:34.906416 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:34.908050 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:13:34.908050 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:13:34.908050 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:34.908050 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:34.908050 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 436/856] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:13:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:34.908050 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9fb3a0-a3cc-4404-9174-adcbd635295d None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:34.908781 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42864f92-f6d7-4ded-bea4-69648b174cad None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:34.909357 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42864f92-f6d7-4ded-bea4-69648b174cad None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:34.909878 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42864f92-f6d7-4ded-bea4-69648b174cad None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:34.911223 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42864f92-f6d7-4ded-bea4-69648b174cad None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:34.914102 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42864f92-f6d7-4ded-bea4-69648b174cad None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:34.943110 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-42864f92-f6d7-4ded-bea4-69648b174cad None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 21 18:13:34.943110 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:34.943110 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:13:34.943110 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:13:34.943110 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:13:34.943110 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:13:34.943110 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:13:34.943110 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:13:34.943110 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 21 18:13:34.943110 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:34.943110 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:13:34.943110 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:34.943110 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:34.943110 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:34.943110 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:34.943110 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:34.945032 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:34.945032 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:34.945032 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:34.945032 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:34.945032 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:34.945032 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:34.945032 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:34.945032 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:13:34.945032 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:13:34.945032 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:13:34.945032 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:13:34.945032 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:34.945032 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:34.945032 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:34.945032 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:34.945032 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:34.946754 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:34.946754 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:34.946754 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:34.946754 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:34.946754 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:34.946754 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:34.946754 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:34.946754 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:34.946754 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:34.946754 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:34.946754 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:34.946754 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:13:34.946754 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:13:34.946754 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 21 18:13:34.946754 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:34.957073 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 421/857] 10.222.0.87 () {56 vars in 1132 bytes} [Tue Jun 21 18:13:34 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:34.957073 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42864f92-f6d7-4ded-bea4-69648b174cad None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:34.977163 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:35.107261 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9fb3a0-a3cc-4404-9174-adcbd635295d None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:35.178866 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66c47d2-8d1b-4205-8c91-76915dda9a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:35.179674 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c47d2-8d1b-4205-8c91-76915dda9a76 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:35.179965 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c47d2-8d1b-4205-8c91-76915dda9a76 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:35.180549 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c47d2-8d1b-4205-8c91-76915dda9a76 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:35.181439 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66c47d2-8d1b-4205-8c91-76915dda9a76 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:35.182268 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66c47d2-8d1b-4205-8c91-76915dda9a76 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:35.183805 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66c47d2-8d1b-4205-8c91-76915dda9a76 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:35.193364 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42864f92-f6d7-4ded-bea4-69648b174cad None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:35.218495 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 437/858] 10.222.0.87 () {56 vars in 1149 bytes} [Tue Jun 21 18:13:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:35.218495 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66c47d2-8d1b-4205-8c91-76915dda9a76 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:35.271066 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77887f7-2ab4-418e-afc7-a3023a1bf8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:35.272281 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77887f7-2ab4-418e-afc7-a3023a1bf8f7 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:35.272815 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77887f7-2ab4-418e-afc7-a3023a1bf8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:35.273293 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77887f7-2ab4-418e-afc7-a3023a1bf8f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:35.274756 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77887f7-2ab4-418e-afc7-a3023a1bf8f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:35.275302 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77887f7-2ab4-418e-afc7-a3023a1bf8f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:35.277703 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77887f7-2ab4-418e-afc7-a3023a1bf8f7 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:35.280606 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:35.305675 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 422/859] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:13:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:35.307185 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77887f7-2ab4-418e-afc7-a3023a1bf8f7 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:35.407091 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:35.413313 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66c47d2-8d1b-4205-8c91-76915dda9a76 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:35.478466 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bec2b3c-f2ac-4433-b282-d0c41e306ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:35.479883 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec2b3c-f2ac-4433-b282-d0c41e306ac9 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:35.480176 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec2b3c-f2ac-4433-b282-d0c41e306ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:35.480461 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec2b3c-f2ac-4433-b282-d0c41e306ac9 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:35.481209 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bec2b3c-f2ac-4433-b282-d0c41e306ac9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:35.481516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bec2b3c-f2ac-4433-b282-d0c41e306ac9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:35.482965 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bec2b3c-f2ac-4433-b282-d0c41e306ac9 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:35.488530 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 438/860] 10.222.0.87 () {56 vars in 1126 bytes} [Tue Jun 21 18:13:35 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:35.489140 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bec2b3c-f2ac-4433-b282-d0c41e306ac9 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:35.493986 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77887f7-2ab4-418e-afc7-a3023a1bf8f7 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:35.552972 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9647b43c-be1b-49a7-b74f-0e7dd310bca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:35.554293 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647b43c-be1b-49a7-b74f-0e7dd310bca9 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:35.554794 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647b43c-be1b-49a7-b74f-0e7dd310bca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:35.555530 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647b43c-be1b-49a7-b74f-0e7dd310bca9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:35.672296 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:35.680050 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bec2b3c-f2ac-4433-b282-d0c41e306ac9 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:35.686937 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9647b43c-be1b-49a7-b74f-0e7dd310bca9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:35.688669 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9647b43c-be1b-49a7-b74f-0e7dd310bca9 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:35.752240 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17731a19-a606-4adb-a555-9a6c0dd015f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:35.753755 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17731a19-a606-4adb-a555-9a6c0dd015f0 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:35.754259 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17731a19-a606-4adb-a555-9a6c0dd015f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:35.754759 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17731a19-a606-4adb-a555-9a6c0dd015f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:35.769117 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17731a19-a606-4adb-a555-9a6c0dd015f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:35.771968 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17731a19-a606-4adb-a555-9a6c0dd015f0 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:35.793743 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-17731a19-a606-4adb-a555-9a6c0dd015f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:13:35.793743 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:35.793743 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:35.793743 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:35.793743 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:35.793743 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:35.793743 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:35.793743 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:35.793743 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:35.793743 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:35.793743 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:35.793743 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:35.793743 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:13:35.793743 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:13:35.793743 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:13:35.795684 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:13:35.795684 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:35.795684 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:35.795684 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:35.795684 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:35.795684 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:35.795684 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:35.795684 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:35.795684 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:35.795684 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:35.795684 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:35.795684 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:35.795684 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:35.795684 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:35.795684 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:35.795684 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:35.795684 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:35.798716 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:13:35.798716 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:13:35.798716 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:13:35.798716 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:13:35.798716 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:13:35.798716 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:13:35.798716 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:13:35.798716 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:35.798716 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 439/861] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:13:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:35.798716 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17731a19-a606-4adb-a555-9a6c0dd015f0 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:35.852950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:35.886921 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 423/862] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:13:35 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 580 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:13:35.895207 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9647b43c-be1b-49a7-b74f-0e7dd310bca9 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:35.999260 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17731a19-a606-4adb-a555-9a6c0dd015f0 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:36.056000 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:36.062182 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ac97f-bb75-45de-9ae1-b9991fbade19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:36.063161 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ac97f-bb75-45de-9ae1-b9991fbade19 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:36.063538 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ac97f-bb75-45de-9ae1-b9991fbade19 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:36.063989 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ac97f-bb75-45de-9ae1-b9991fbade19 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:36.064893 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10ac97f-bb75-45de-9ae1-b9991fbade19 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:36.065297 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10ac97f-bb75-45de-9ae1-b9991fbade19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:36.066416 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9647b43c-be1b-49a7-b74f-0e7dd310bca9 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:36.066990 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10ac97f-bb75-45de-9ae1-b9991fbade19 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:36.075298 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 440/863] 10.222.0.87 () {56 vars in 1126 bytes} [Tue Jun 21 18:13:35 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:36.076347 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ac97f-bb75-45de-9ae1-b9991fbade19 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:36.142084 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c03d7fb-0b88-42ca-9f37-d0ccc861337f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:36.142788 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c03d7fb-0b88-42ca-9f37-d0ccc861337f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:36.143074 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c03d7fb-0b88-42ca-9f37-d0ccc861337f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:36.143357 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c03d7fb-0b88-42ca-9f37-d0ccc861337f None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:36.144180 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c03d7fb-0b88-42ca-9f37-d0ccc861337f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:36.144498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c03d7fb-0b88-42ca-9f37-d0ccc861337f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:36.146068 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c03d7fb-0b88-42ca-9f37-d0ccc861337f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:36.163312 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 424/864] 10.222.0.87 () {56 vars in 1148 bytes} [Tue Jun 21 18:13:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:36.165416 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c24b38c-682e-4264-a1fd-b3577bb72809 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:36.165774 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c24b38c-682e-4264-a1fd-b3577bb72809 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:36.166152 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c24b38c-682e-4264-a1fd-b3577bb72809 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:36.173778 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 425/865] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:13:36 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:13:36.174813 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c24b38c-682e-4264-a1fd-b3577bb72809 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:36.281943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:36.307112 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ac97f-bb75-45de-9ae1-b9991fbade19 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:36.373714 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0d9645-d33a-4e30-85d2-1180715f511a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:36.375324 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d9645-d33a-4e30-85d2-1180715f511a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:36.375926 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d9645-d33a-4e30-85d2-1180715f511a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:36.376520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d9645-d33a-4e30-85d2-1180715f511a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:36.381871 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c24b38c-682e-4264-a1fd-b3577bb72809 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:36.397885 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0d9645-d33a-4e30-85d2-1180715f511a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:36.400994 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0d9645-d33a-4e30-85d2-1180715f511a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:36.436030 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-0a0d9645-d33a-4e30-85d2-1180715f511a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:36.436030 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:36.436030 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:36.436030 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:36.436030 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:36.436030 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:36.436030 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:36.436030 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:36.436030 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:36.436030 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:36.436030 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:36.436030 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:36.436030 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:13:36.436030 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:13:36.436030 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:13:36.437288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:13:36.437288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:36.437288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:36.437288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:36.437288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:36.437288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:36.437288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:36.437288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:36.437288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:36.437288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:36.437288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:36.437288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:36.437288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:36.437288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:36.437288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:36.437288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:36.437288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:36.438329 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:13:36.438329 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:13:36.438329 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:36.438329 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:36.445865 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 441/866] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:13:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:36.446286 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb9ecf5-6c9b-4112-8eea-950158ad13bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:36.446997 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9ecf5-6c9b-4112-8eea-950158ad13bf None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:36.447291 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9ecf5-6c9b-4112-8eea-950158ad13bf None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:36.447556 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9ecf5-6c9b-4112-8eea-950158ad13bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:36.448142 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfefb7d-9678-4df1-b1e4-609edd665c17 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:36.448425 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb9ecf5-6c9b-4112-8eea-950158ad13bf None admin] RBAC: Adding query filter params (role.id=98f0943a610f4a9e9f8de54f6e9f717d, scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb, user.id=515b3571045e4fe29565307c8ecc10b2) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:36.448674 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfefb7d-9678-4df1-b1e4-609edd665c17 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:36.448825 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb9ecf5-6c9b-4112-8eea-950158ad13bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:36.449114 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfefb7d-9678-4df1-b1e4-609edd665c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:36.450274 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb9ecf5-6c9b-4112-8eea-950158ad13bf None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:36.461930 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 426/867] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:13:36 2022] GET /identity/v3/role_assignments?user.id=515b3571045e4fe29565307c8ecc10b2&scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb&role.id=98f0943a610f4a9e9f8de54f6e9f717d => generated 608 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:36.464277 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb9ecf5-6c9b-4112-8eea-950158ad13bf None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:36.480233 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-6bfefb7d-9678-4df1-b1e4-609edd665c17 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:13:36.482565 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-6bfefb7d-9678-4df1-b1e4-609edd665c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:13:36.525267 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:36.653785 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 442/868] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:13:36 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:13:36.687620 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfefb7d-9678-4df1-b1e4-609edd665c17 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:36.755069 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb9ecf5-6c9b-4112-8eea-950158ad13bf None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:36.850653 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2330029c-3842-48ca-9676-dae1bd14e16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:36.852107 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330029c-3842-48ca-9676-dae1bd14e16b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:36.852586 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330029c-3842-48ca-9676-dae1bd14e16b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:36.853097 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330029c-3842-48ca-9676-dae1bd14e16b None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:36.855204 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2330029c-3842-48ca-9676-dae1bd14e16b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:36.855760 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2330029c-3842-48ca-9676-dae1bd14e16b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:36.858951 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2330029c-3842-48ca-9676-dae1bd14e16b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:36.881629 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:36.900495 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 427/869] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:13:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 437 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:36.909937 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2330029c-3842-48ca-9676-dae1bd14e16b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:36.967012 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfefb7d-9678-4df1-b1e4-609edd665c17 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:37.057432 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1089c4a7-02b9-467b-a34c-1ed988b650ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:37.058636 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1089c4a7-02b9-467b-a34c-1ed988b650ae None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:37.059100 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1089c4a7-02b9-467b-a34c-1ed988b650ae None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:37.059842 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1089c4a7-02b9-467b-a34c-1ed988b650ae None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:37.061232 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1089c4a7-02b9-467b-a34c-1ed988b650ae None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:37.063394 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1089c4a7-02b9-467b-a34c-1ed988b650ae None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:37.088229 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 443/870] 10.222.0.87 () {60 vars in 1141 bytes} [Tue Jun 21 18:13:36 2022] POST /identity/v3/services => generated 251 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:13:37.102344 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:37.166526 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2330029c-3842-48ca-9676-dae1bd14e16b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:37.264827 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b395be-ad88-43e4-88b3-7b475904c560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:37.266227 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b395be-ad88-43e4-88b3-7b475904c560 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:37.266685 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b395be-ad88-43e4-88b3-7b475904c560 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:37.267126 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b395be-ad88-43e4-88b3-7b475904c560 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:37.280462 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b395be-ad88-43e4-88b3-7b475904c560 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:37.283293 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b395be-ad88-43e4-88b3-7b475904c560 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:37.316969 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-e4b395be-ad88-43e4-88b3-7b475904c560 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:13:37.316969 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:37.316969 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:37.316969 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:37.316969 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:37.316969 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:37.316969 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:37.316969 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:37.316969 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:37.316969 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:37.316969 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:37.316969 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:37.316969 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:13:37.316969 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:13:37.316969 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:13:37.320871 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:13:37.320871 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:37.320871 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:37.320871 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:37.320871 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:37.320871 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:37.320871 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:37.320871 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:37.320871 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:37.320871 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:37.320871 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:37.320871 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:37.320871 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:37.320871 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:37.320871 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:37.320871 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:37.320871 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:37.323454 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:13:37.323454 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:13:37.323454 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:13:37.323454 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:13:37.323454 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:13:37.323454 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:13:37.323454 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:13:37.323454 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:37.323454 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 428/871] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:13:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 411 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:37.329678 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1089c4a7-02b9-467b-a34c-1ed988b650ae None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:37.526472 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1089c4a7-02b9-467b-a34c-1ed988b650ae None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:37.579698 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:37.589821 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0729bc8d-28e6-4906-b3f0-05028976b7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:37.590997 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0729bc8d-28e6-4906-b3f0-05028976b7cf None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:37.591863 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0729bc8d-28e6-4906-b3f0-05028976b7cf None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:37.592355 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0729bc8d-28e6-4906-b3f0-05028976b7cf None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:37.593906 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0729bc8d-28e6-4906-b3f0-05028976b7cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:37.595761 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0729bc8d-28e6-4906-b3f0-05028976b7cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:37.598357 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0729bc8d-28e6-4906-b3f0-05028976b7cf None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:37.622928 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 444/872] 10.222.0.87 () {56 vars in 1148 bytes} [Tue Jun 21 18:13:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:37.631683 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b395be-ad88-43e4-88b3-7b475904c560 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:37.860037 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b395be-ad88-43e4-88b3-7b475904c560 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:37.921369 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9a5238-1243-4a9a-8b30-6d12d59b6f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:37.922777 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a5238-1243-4a9a-8b30-6d12d59b6f0b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:37.923270 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a5238-1243-4a9a-8b30-6d12d59b6f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:37.923729 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a5238-1243-4a9a-8b30-6d12d59b6f0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:37.925333 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9a5238-1243-4a9a-8b30-6d12d59b6f0b None admin] RBAC: Adding query filter params (role.id=b9dbf55e6f2947158c1d946f6ee9f907, scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb, user.id=09c9fa7dccb8491c9ad3b43967345ea2) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:37.925890 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9a5238-1243-4a9a-8b30-6d12d59b6f0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:37.929223 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9a5238-1243-4a9a-8b30-6d12d59b6f0b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:37.941299 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:37.945572 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 429/873] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:13:37 2022] GET /identity/v3/role_assignments?user.id=09c9fa7dccb8491c9ad3b43967345ea2&scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb&role.id=b9dbf55e6f2947158c1d946f6ee9f907 => generated 608 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:39.542310 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3bc8fc-95b2-4489-80f4-7dfdba1058a3 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:39.543112 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3bc8fc-95b2-4489-80f4-7dfdba1058a3 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:39.543247 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3bc8fc-95b2-4489-80f4-7dfdba1058a3 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:39.546201 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 445/874] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:13:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:13:39.557241 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5496cb1-a99e-45c2-b889-84cf00e006a4 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:39.558826 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5496cb1-a99e-45c2-b889-84cf00e006a4 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:39.559642 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5496cb1-a99e-45c2-b889-84cf00e006a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:39.591985 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b5496cb1-a99e-45c2-b889-84cf00e006a4 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:13:39.594594 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b5496cb1-a99e-45c2-b889-84cf00e006a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:13:39.753756 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 430/875] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:13:39 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:13:39.795002 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3bc8fc-95b2-4489-80f4-7dfdba1058a3 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:40.023497 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3bc8fc-95b2-4489-80f4-7dfdba1058a3 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:40.082883 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebf62c-541e-40ac-8704-4dbf2a218fe3 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:40.083527 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebf62c-541e-40ac-8704-4dbf2a218fe3 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:40.084033 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebf62c-541e-40ac-8704-4dbf2a218fe3 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:40.086232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 431/876] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:13:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:13:40.094374 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28398d-ad9d-46ff-9f41-59216a5d0fe2 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:40.095152 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28398d-ad9d-46ff-9f41-59216a5d0fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:40.095641 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2a8cfa-46d8-4315-acb0-697d61e07e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:40.096470 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28398d-ad9d-46ff-9f41-59216a5d0fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:40.097082 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a8cfa-46d8-4315-acb0-697d61e07e20 None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:40.097608 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a8cfa-46d8-4315-acb0-697d61e07e20 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:40.098085 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a8cfa-46d8-4315-acb0-697d61e07e20 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:40.099606 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2a8cfa-46d8-4315-acb0-697d61e07e20 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:40.102575 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2a8cfa-46d8-4315-acb0-697d61e07e20 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:40.130380 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-9c28398d-ad9d-46ff-9f41-59216a5d0fe2 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:13:40.130664 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 446/877] 10.222.0.87 () {60 vars in 1141 bytes} [Tue Jun 21 18:13:39 2022] POST /identity/v3/services => generated 269 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:13:40.133117 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9c28398d-ad9d-46ff-9f41-59216a5d0fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:13:40.266844 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 432/878] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:13:40 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:13:40.308063 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2a8cfa-46d8-4315-acb0-697d61e07e20 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:40.437185 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:40.555081 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2a8cfa-46d8-4315-acb0-697d61e07e20 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:40.640453 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87ec8f4-a89f-4d86-a46d-ec59cd12a089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:40.642821 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ec8f4-a89f-4d86-a46d-ec59cd12a089 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:40.643865 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ec8f4-a89f-4d86-a46d-ec59cd12a089 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:40.644784 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ec8f4-a89f-4d86-a46d-ec59cd12a089 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:40.646650 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87ec8f4-a89f-4d86-a46d-ec59cd12a089 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:40.650212 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87ec8f4-a89f-4d86-a46d-ec59cd12a089 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:40.676886 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-a87ec8f4-a89f-4d86-a46d-ec59cd12a089 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 18:13:40.676886 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:40.676886 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:13:40.676886 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:13:40.676886 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:13:40.676886 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:13:40.676886 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:13:40.676886 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:13:40.676886 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 18:13:40.676886 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:40.676886 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:13:40.676886 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:40.676886 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:40.676886 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:40.676886 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:40.676886 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:40.676886 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:40.678967 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:40.678967 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:40.678967 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:40.678967 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:40.678967 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:40.678967 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:40.678967 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:13:40.678967 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:13:40.678967 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:13:40.678967 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:13:40.678967 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:40.678967 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:40.678967 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:40.678967 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:40.678967 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:40.678967 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:40.680661 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:40.680661 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:40.680661 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:40.680661 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:40.680661 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:40.680661 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:40.680661 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:40.680661 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:40.680661 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:40.680661 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:40.680661 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:13:40.680661 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:13:40.680661 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 18:13:40.680661 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:40.686291 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 447/879] 10.222.0.87 () {56 vars in 1126 bytes} [Tue Jun 21 18:13:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:40.698542 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c28398d-ad9d-46ff-9f41-59216a5d0fe2 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:40.822787 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:40.889540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c28398d-ad9d-46ff-9f41-59216a5d0fe2 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:40.922535 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d9a9b-5b7a-40da-8c64-406e85581110 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:40.923541 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d9a9b-5b7a-40da-8c64-406e85581110 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:40.924316 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d9a9b-5b7a-40da-8c64-406e85581110 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:40.933838 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 448/880] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:13:40 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:13:40.937547 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0d001-8551-4a34-921d-2a48e0b926de None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:40.938534 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0d001-8551-4a34-921d-2a48e0b926de None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:40.939394 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0d001-8551-4a34-921d-2a48e0b926de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:40.945774 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4504e800-08d7-4410-ba9b-62313edb7193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:40.946942 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504e800-08d7-4410-ba9b-62313edb7193 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:40.947464 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504e800-08d7-4410-ba9b-62313edb7193 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:40.947958 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504e800-08d7-4410-ba9b-62313edb7193 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:40.949358 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4504e800-08d7-4410-ba9b-62313edb7193 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:40.949825 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4504e800-08d7-4410-ba9b-62313edb7193 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:40.951528 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4504e800-08d7-4410-ba9b-62313edb7193 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:40.958334 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 433/881] 10.222.0.87 () {56 vars in 1122 bytes} [Tue Jun 21 18:13:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:40.966460 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4504e800-08d7-4410-ba9b-62313edb7193 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:40.979397 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-a2f0d001-8551-4a34-921d-2a48e0b926de None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:13:40.981992 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a2f0d001-8551-4a34-921d-2a48e0b926de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:13:41.142199 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 449/882] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:13:40 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:13:41.153555 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4504e800-08d7-4410-ba9b-62313edb7193 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:41.188954 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f0d001-8551-4a34-921d-2a48e0b926de None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:41.198505 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:41.225335 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62bf29b-dab1-48b1-9a16-ba58235a67d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:41.225903 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62bf29b-dab1-48b1-9a16-ba58235a67d0 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:41.227165 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62bf29b-dab1-48b1-9a16-ba58235a67d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:41.227665 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62bf29b-dab1-48b1-9a16-ba58235a67d0 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:41.228592 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62bf29b-dab1-48b1-9a16-ba58235a67d0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:41.228919 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62bf29b-dab1-48b1-9a16-ba58235a67d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:41.231008 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62bf29b-dab1-48b1-9a16-ba58235a67d0 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:41.245696 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 434/883] 10.222.0.87 () {56 vars in 1122 bytes} [Tue Jun 21 18:13:40 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:13:41.301005 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:41.392696 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f0d001-8551-4a34-921d-2a48e0b926de None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:41.449025 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139c710f-65f8-45e4-9b5d-fb1bc4e8f626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:41.449972 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c710f-65f8-45e4-9b5d-fb1bc4e8f626 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:41.450268 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c710f-65f8-45e4-9b5d-fb1bc4e8f626 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:41.450546 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c710f-65f8-45e4-9b5d-fb1bc4e8f626 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:41.451291 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139c710f-65f8-45e4-9b5d-fb1bc4e8f626 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:41.452753 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139c710f-65f8-45e4-9b5d-fb1bc4e8f626 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:41.468236 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-139c710f-65f8-45e4-9b5d-fb1bc4e8f626 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 21 18:13:41.468236 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:41.468236 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:13:41.468236 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:13:41.468236 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:13:41.468236 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:13:41.468236 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:13:41.468236 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:13:41.468236 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 21 18:13:41.468236 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:41.468236 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:13:41.468236 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:41.468236 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:41.468236 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:41.468236 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:41.468236 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:41.468236 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:41.470642 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:41.470642 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:41.470642 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:41.470642 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:41.470642 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:41.470642 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:41.470642 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:13:41.470642 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:13:41.470642 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:13:41.470642 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:13:41.470642 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:41.470642 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:41.470642 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:41.470642 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:41.470642 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:41.470642 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:41.472755 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:41.472755 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:41.472755 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:41.472755 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:41.472755 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:41.472755 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:41.472755 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:41.472755 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:41.472755 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:41.472755 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:41.472755 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:13:41.472755 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:13:41.472755 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 21 18:13:41.472755 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:41.472755 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 450/884] 10.222.0.87 () {56 vars in 1114 bytes} [Tue Jun 21 18:13:41 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:41.486017 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62bf29b-dab1-48b1-9a16-ba58235a67d0 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:41.630254 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ccb2e-d4e8-4965-be49-430cf0446f90 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:41.632955 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ccb2e-d4e8-4965-be49-430cf0446f90 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:41.633785 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ccb2e-d4e8-4965-be49-430cf0446f90 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:41.636714 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 451/885] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:13:41 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:13:41.646324 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40fe720-d38c-44a7-9d94-17710058b51d None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:41.647276 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40fe720-d38c-44a7-9d94-17710058b51d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:41.648027 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40fe720-d38c-44a7-9d94-17710058b51d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:41.680102 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-b40fe720-d38c-44a7-9d94-17710058b51d None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:13:41.682665 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b40fe720-d38c-44a7-9d94-17710058b51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:13:41.722157 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62bf29b-dab1-48b1-9a16-ba58235a67d0 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:41.807870 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf41534-f636-4a4a-8ee8-f3e35acae6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:41.820543 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf41534-f636-4a4a-8ee8-f3e35acae6a0 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:41.821469 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf41534-f636-4a4a-8ee8-f3e35acae6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:41.822436 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf41534-f636-4a4a-8ee8-f3e35acae6a0 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:41.824623 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf41534-f636-4a4a-8ee8-f3e35acae6a0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:41.825945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf41534-f636-4a4a-8ee8-f3e35acae6a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:41.829832 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf41534-f636-4a4a-8ee8-f3e35acae6a0 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:41.844405 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 452/886] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:13:41 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:13:41.851791 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 435/887] 10.222.0.87 () {56 vars in 1114 bytes} [Tue Jun 21 18:13:41 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:13:41.865409 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40fe720-d38c-44a7-9d94-17710058b51d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:41.883029 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf41534-f636-4a4a-8ee8-f3e35acae6a0 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:41.950919 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:41.999971 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:42.076325 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40fe720-d38c-44a7-9d94-17710058b51d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:42.133309 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf41534-f636-4a4a-8ee8-f3e35acae6a0 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:42.144162 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb716fa-d9e2-42dc-9484-d0f8d5009a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:42.145919 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb716fa-d9e2-42dc-9484-d0f8d5009a07 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:42.146906 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb716fa-d9e2-42dc-9484-d0f8d5009a07 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:42.147711 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb716fa-d9e2-42dc-9484-d0f8d5009a07 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:42.149371 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb716fa-d9e2-42dc-9484-d0f8d5009a07 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:42.150333 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb716fa-d9e2-42dc-9484-d0f8d5009a07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:42.153297 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb716fa-d9e2-42dc-9484-d0f8d5009a07 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:42.162529 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 453/888] 10.222.0.87 () {56 vars in 1114 bytes} [Tue Jun 21 18:13:41 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:42.180074 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb716fa-d9e2-42dc-9484-d0f8d5009a07 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:42.230615 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aee4ba-5947-4134-a2a7-ab02d20b5f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:42.232633 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aee4ba-5947-4134-a2a7-ab02d20b5f11 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:42.233337 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aee4ba-5947-4134-a2a7-ab02d20b5f11 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:42.234089 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aee4ba-5947-4134-a2a7-ab02d20b5f11 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:42.258508 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aee4ba-5947-4134-a2a7-ab02d20b5f11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:13:42.274037 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-c4aee4ba-5947-4134-a2a7-ab02d20b5f11 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 18:13:42.274037 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:42.274037 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:42.274037 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:42.274037 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:42.274037 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:42.274037 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:42.274037 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:42.274037 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:42.274037 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:42.274037 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:42.274037 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:42.274037 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:13:42.274037 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:13:42.274037 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:13:42.274037 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:13:42.275654 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:13:42.275654 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:13:42.275654 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:13:42.275654 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:13:42.275654 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:42.275654 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:42.275654 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:42.275654 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:42.275654 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:42.275654 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:42.275654 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:42.275654 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:42.275654 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:42.275654 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:42.275654 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:42.275654 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:42.276597 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:42.276597 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:42.276597 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:42.276597 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:42.276597 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:13:42.276597 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:13:42.276597 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:13:42.276597 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:13:42.276597 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:13:42.276597 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:13:42.276597 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 18:13:42.276597 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:42.286366 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 436/889] 10.222.0.87 () {56 vars in 1120 bytes} [Tue Jun 21 18:13:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 398 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:13:42.287976 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aee4ba-5947-4134-a2a7-ab02d20b5f11 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:42.441527 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb716fa-d9e2-42dc-9484-d0f8d5009a07 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:42.444434 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:42.531871 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46026d68-2a43-4ff7-bb80-927f7058d466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:42.534351 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46026d68-2a43-4ff7-bb80-927f7058d466 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:42.535362 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46026d68-2a43-4ff7-bb80-927f7058d466 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:42.536511 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46026d68-2a43-4ff7-bb80-927f7058d466 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:42.538430 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46026d68-2a43-4ff7-bb80-927f7058d466 None admin] RBAC: Adding query filter params (interface=public, service_id=768f3c89e6dd4b52b0132c1bb13425d2, region_id=RegionOne) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:42.539965 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46026d68-2a43-4ff7-bb80-927f7058d466 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:42.543752 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46026d68-2a43-4ff7-bb80-927f7058d466 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:42.554377 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aee4ba-5947-4134-a2a7-ab02d20b5f11 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:42.566344 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 454/890] 10.222.0.87 () {56 vars in 1251 bytes} [Tue Jun 21 18:13:42 2022] GET /identity/v3/endpoints?service_id=768f3c89e6dd4b52b0132c1bb13425d2&interface=public®ion_id=RegionOne => generated 194 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:42.572628 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:42.576326 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46026d68-2a43-4ff7-bb80-927f7058d466 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:42.638371 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b26455-3692-4408-9e47-ba529fd76f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:42.638371 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b26455-3692-4408-9e47-ba529fd76f5b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:42.638371 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b26455-3692-4408-9e47-ba529fd76f5b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:42.638371 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b26455-3692-4408-9e47-ba529fd76f5b None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:42.640091 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b26455-3692-4408-9e47-ba529fd76f5b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:42.640091 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b26455-3692-4408-9e47-ba529fd76f5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:42.642220 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b26455-3692-4408-9e47-ba529fd76f5b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:42.660476 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 437/891] 10.222.0.87 () {56 vars in 1115 bytes} [Tue Jun 21 18:13:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:42.672622 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b26455-3692-4408-9e47-ba529fd76f5b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:42.832964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46026d68-2a43-4ff7-bb80-927f7058d466 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:42.901441 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b26455-3692-4408-9e47-ba529fd76f5b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:42.924625 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde3fe49-00ab-404c-9978-8653adc0dc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:42.926516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3fe49-00ab-404c-9978-8653adc0dc57 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:42.927626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3fe49-00ab-404c-9978-8653adc0dc57 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:42.929656 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3fe49-00ab-404c-9978-8653adc0dc57 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:42.931603 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde3fe49-00ab-404c-9978-8653adc0dc57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:42.935059 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde3fe49-00ab-404c-9978-8653adc0dc57 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:42.954336 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 455/892] 10.222.0.87 () {56 vars in 1087 bytes} [Tue Jun 21 18:13:42 2022] GET /identity/v3/services => generated 1268 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:13:42.981476 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c37f7c8-49bb-4dc2-989e-894ebe31ec0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:42.982829 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c37f7c8-49bb-4dc2-989e-894ebe31ec0c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:42.983281 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c37f7c8-49bb-4dc2-989e-894ebe31ec0c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:42.983715 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c37f7c8-49bb-4dc2-989e-894ebe31ec0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:43.008606 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c37f7c8-49bb-4dc2-989e-894ebe31ec0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:43.012364 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c37f7c8-49bb-4dc2-989e-894ebe31ec0c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:43.053147 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-2c37f7c8-49bb-4dc2-989e-894ebe31ec0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:43.053147 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:43.053147 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:43.053147 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:43.053147 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:43.053147 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:43.053147 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:43.053147 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:43.053147 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:43.053147 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:43.053147 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:43.053147 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:43.053147 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:13:43.053147 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:13:43.053147 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:13:43.055676 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:13:43.055676 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:43.055676 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:43.055676 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:43.055676 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:43.055676 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:43.055676 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:43.055676 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:43.055676 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:43.055676 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:43.055676 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:43.055676 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:43.055676 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:43.055676 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:43.055676 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:43.055676 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:43.055676 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:43.058664 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:13:43.058664 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:13:43.058664 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:43.058664 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:43.058664 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 438/893] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:13:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:13:43.067149 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde3fe49-00ab-404c-9978-8653adc0dc57 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:43.141146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:43.305755 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde3fe49-00ab-404c-9978-8653adc0dc57 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:43.387179 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3289d3-8151-425c-8a88-dd5dce06bc26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:43.388910 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3289d3-8151-425c-8a88-dd5dce06bc26 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:43.389750 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3289d3-8151-425c-8a88-dd5dce06bc26 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:43.390514 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3289d3-8151-425c-8a88-dd5dce06bc26 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:43.392196 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3289d3-8151-425c-8a88-dd5dce06bc26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:43.393064 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3289d3-8151-425c-8a88-dd5dce06bc26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:43.396196 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3289d3-8151-425c-8a88-dd5dce06bc26 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:43.422938 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 456/894] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:13:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:43.431722 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c37f7c8-49bb-4dc2-989e-894ebe31ec0c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:43.479859 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:43.661793 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c37f7c8-49bb-4dc2-989e-894ebe31ec0c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:43.745360 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb558c93-73dd-4d55-b94e-db976d8fc622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:43.747380 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb558c93-73dd-4d55-b94e-db976d8fc622 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:43.748171 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb558c93-73dd-4d55-b94e-db976d8fc622 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:43.748937 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb558c93-73dd-4d55-b94e-db976d8fc622 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:43.766882 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb558c93-73dd-4d55-b94e-db976d8fc622 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:43.769671 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb558c93-73dd-4d55-b94e-db976d8fc622 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:43.800471 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-eb558c93-73dd-4d55-b94e-db976d8fc622 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 18:13:43.800471 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:43.800471 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:43.800471 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:43.800471 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:43.800471 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:43.800471 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:43.800471 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:43.800471 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:43.800471 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:43.800471 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:43.800471 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:43.800471 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:13:43.800471 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:13:43.800471 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:13:43.800471 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:13:43.802310 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:43.802310 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:43.802310 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:13:43.802310 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:13:43.802310 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:13:43.802310 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:13:43.802310 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:43.802310 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:43.802310 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:43.802310 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:43.802310 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:43.802310 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:43.802310 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:43.802310 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:43.802310 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:43.802310 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:43.802310 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:43.802310 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:43.805033 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:43.805033 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:43.805033 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:13:43.805033 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:13:43.805033 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:13:43.805033 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:13:43.805033 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:13:43.805033 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:13:43.805033 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 18:13:43.805033 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:43.805033 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 439/895] 10.222.0.87 () {56 vars in 1117 bytes} [Tue Jun 21 18:13:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:43.814916 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3289d3-8151-425c-8a88-dd5dce06bc26 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:43.919373 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6557f92-7275-43ea-a695-2fbd473bb459 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:43.920125 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6557f92-7275-43ea-a695-2fbd473bb459 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:43.920252 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6557f92-7275-43ea-a695-2fbd473bb459 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:43.923536 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 440/896] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:13:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:13:43.932108 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7485f0-21fa-4616-ba41-3adef0421fda None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:43.932563 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7485f0-21fa-4616-ba41-3adef0421fda None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:43.933025 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7485f0-21fa-4616-ba41-3adef0421fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:43.979886 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-6b7485f0-21fa-4616-ba41-3adef0421fda None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:13:43.981905 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6b7485f0-21fa-4616-ba41-3adef0421fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:13:44.020824 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:44.044587 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3289d3-8151-425c-8a88-dd5dce06bc26 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:44.112064 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86008581-af23-47b3-a430-370f4ae28883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:44.113619 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86008581-af23-47b3-a430-370f4ae28883 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:44.114479 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86008581-af23-47b3-a430-370f4ae28883 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:44.116275 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86008581-af23-47b3-a430-370f4ae28883 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:44.118586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86008581-af23-47b3-a430-370f4ae28883 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:44.119459 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86008581-af23-47b3-a430-370f4ae28883 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:44.122815 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86008581-af23-47b3-a430-370f4ae28883 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:44.147743 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 441/897] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:13:43 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:13:44.163149 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 457/898] 10.222.0.87 () {56 vars in 1149 bytes} [Tue Jun 21 18:13:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:44.171930 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7485f0-21fa-4616-ba41-3adef0421fda None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:44.186576 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86008581-af23-47b3-a430-370f4ae28883 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:44.196923 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:44.405474 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7485f0-21fa-4616-ba41-3adef0421fda None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:44.418003 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86008581-af23-47b3-a430-370f4ae28883 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:44.490217 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e0065-4db8-4ed1-9b5e-37a3bab0ff2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:44.492065 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e0065-4db8-4ed1-9b5e-37a3bab0ff2f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:44.492758 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e0065-4db8-4ed1-9b5e-37a3bab0ff2f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:44.493433 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e0065-4db8-4ed1-9b5e-37a3bab0ff2f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:44.494215 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5008f996-fa0b-412e-9156-2c83c725ce8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:44.494873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e0065-4db8-4ed1-9b5e-37a3bab0ff2f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:44.495316 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008f996-fa0b-412e-9156-2c83c725ce8a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:44.495807 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008f996-fa0b-412e-9156-2c83c725ce8a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:44.496292 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008f996-fa0b-412e-9156-2c83c725ce8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:44.497957 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e0065-4db8-4ed1-9b5e-37a3bab0ff2f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:44.525535 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-357e0065-4db8-4ed1-9b5e-37a3bab0ff2f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 21 18:13:44.525535 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:44.525535 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:13:44.525535 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:13:44.525535 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:13:44.525535 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:13:44.525535 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:13:44.525535 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:13:44.525535 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 21 18:13:44.525535 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:44.525535 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:13:44.525535 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:44.525535 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:44.525535 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:44.525535 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:44.525535 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:44.527335 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:44.527335 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:44.527335 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:44.527335 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:44.527335 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:44.527335 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:44.527335 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:44.527335 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:13:44.527335 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:13:44.527335 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:13:44.527335 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:13:44.527335 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:44.527335 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:44.527335 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:44.527335 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:44.527335 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:44.528950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:44.528950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:44.528950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:44.528950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:44.528950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:44.528950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:44.528950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:44.528950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:44.528950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:44.528950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:44.528950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:44.528950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:13:44.528950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:13:44.528950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 21 18:13:44.528950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:44.528950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 458/899] 10.222.0.87 () {56 vars in 1132 bytes} [Tue Jun 21 18:13:44 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:13:44.542828 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e0065-4db8-4ed1-9b5e-37a3bab0ff2f None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:44.570278 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:44.658994 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5008f996-fa0b-412e-9156-2c83c725ce8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:44.660732 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5008f996-fa0b-412e-9156-2c83c725ce8a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:44.682457 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:44.758623 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e0065-4db8-4ed1-9b5e-37a3bab0ff2f None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:44.821368 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f4a5e7-4d94-48a1-bb39-c654e4ef04b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:44.822345 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4a5e7-4d94-48a1-bb39-c654e4ef04b5 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:44.822714 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4a5e7-4d94-48a1-bb39-c654e4ef04b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:44.823077 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4a5e7-4d94-48a1-bb39-c654e4ef04b5 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:44.824763 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f4a5e7-4d94-48a1-bb39-c654e4ef04b5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:44.825457 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f4a5e7-4d94-48a1-bb39-c654e4ef04b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:44.826787 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f4a5e7-4d94-48a1-bb39-c654e4ef04b5 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:44.841688 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 459/900] 10.222.0.87 () {56 vars in 1126 bytes} [Tue Jun 21 18:13:44 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:44.854002 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bcfe9e-305f-4499-a93f-42229a6d1fa0 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:44.854495 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bcfe9e-305f-4499-a93f-42229a6d1fa0 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:44.854981 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bcfe9e-305f-4499-a93f-42229a6d1fa0 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:44.863420 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 460/901] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:13:44 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:13:44.866091 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bcfe9e-305f-4499-a93f-42229a6d1fa0 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:44.886824 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 442/902] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:13:44 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 717 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:13:44.888403 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b27ab3-ea1b-46a8-892a-918c822be123 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:44.888717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b27ab3-ea1b-46a8-892a-918c822be123 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:44.888979 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b27ab3-ea1b-46a8-892a-918c822be123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:44.926265 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-10b27ab3-ea1b-46a8-892a-918c822be123 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:13:44.928426 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-10b27ab3-ea1b-46a8-892a-918c822be123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:13:45.046817 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:45.056666 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 443/903] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:13:44 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:13:45.056666 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b27ab3-ea1b-46a8-892a-918c822be123 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:45.100075 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bcfe9e-305f-4499-a93f-42229a6d1fa0 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:45.136352 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:45.163891 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedb62c2-5fd5-42ab-9142-7a351a84c704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:45.165412 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb62c2-5fd5-42ab-9142-7a351a84c704 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:45.166501 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb62c2-5fd5-42ab-9142-7a351a84c704 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:45.166951 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb62c2-5fd5-42ab-9142-7a351a84c704 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:45.168284 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedb62c2-5fd5-42ab-9142-7a351a84c704 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:45.168788 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedb62c2-5fd5-42ab-9142-7a351a84c704 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:45.171500 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedb62c2-5fd5-42ab-9142-7a351a84c704 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:45.186228 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 461/904] 10.222.0.87 () {56 vars in 1126 bytes} [Tue Jun 21 18:13:44 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:45.186228 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedb62c2-5fd5-42ab-9142-7a351a84c704 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:45.235921 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b27ab3-ea1b-46a8-892a-918c822be123 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:45.288414 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f9ebb7-5b31-4878-9668-ae02dbc0a2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:45.289320 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f9ebb7-5b31-4878-9668-ae02dbc0a2bd None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:45.289609 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f9ebb7-5b31-4878-9668-ae02dbc0a2bd None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:45.289995 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f9ebb7-5b31-4878-9668-ae02dbc0a2bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:45.304592 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f9ebb7-5b31-4878-9668-ae02dbc0a2bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:45.306643 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f9ebb7-5b31-4878-9668-ae02dbc0a2bd None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:45.329844 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-d7f9ebb7-5b31-4878-9668-ae02dbc0a2bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:45.329844 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:45.329844 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:45.329844 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:45.329844 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:45.329844 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:45.329844 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:45.329844 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:45.329844 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:45.329844 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:45.329844 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:45.329844 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:45.329844 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:13:45.329844 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:13:45.329844 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:13:45.331551 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:13:45.331551 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:45.331551 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:45.331551 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:45.331551 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:45.331551 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:45.331551 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:45.331551 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:45.331551 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:45.331551 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:45.331551 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:45.331551 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:45.331551 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:45.331551 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:45.331551 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:45.331551 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:45.331551 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:45.333253 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:13:45.333253 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:13:45.333253 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:45.333253 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:45.333253 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 444/905] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:13:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:45.333253 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f9ebb7-5b31-4878-9668-ae02dbc0a2bd None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:45.406293 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedb62c2-5fd5-42ab-9142-7a351a84c704 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:45.489182 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12143ac0-80ef-409e-a775-9b60c9fd7a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:45.490834 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12143ac0-80ef-409e-a775-9b60c9fd7a53 None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:45.491588 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12143ac0-80ef-409e-a775-9b60c9fd7a53 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:45.492296 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12143ac0-80ef-409e-a775-9b60c9fd7a53 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:45.494452 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12143ac0-80ef-409e-a775-9b60c9fd7a53 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:45.497561 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12143ac0-80ef-409e-a775-9b60c9fd7a53 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:45.499597 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:45.517864 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:45.521543 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f9ebb7-5b31-4878-9668-ae02dbc0a2bd None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:45.532054 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 462/906] 10.222.0.87 () {60 vars in 1141 bytes} [Tue Jun 21 18:13:45 2022] POST /identity/v3/services => generated 241 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:13:45.533831 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12143ac0-80ef-409e-a775-9b60c9fd7a53 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:45.625440 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82fb02b-9993-4740-bd23-8439427eebd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:45.626981 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82fb02b-9993-4740-bd23-8439427eebd0 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:45.627591 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82fb02b-9993-4740-bd23-8439427eebd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:45.628155 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82fb02b-9993-4740-bd23-8439427eebd0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:45.634006 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82fb02b-9993-4740-bd23-8439427eebd0 None admin] RBAC: Adding query filter params (interface=public, service_id=148a89a6d4284a62a366c6f25ab4837c, region_id=RegionOne) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:45.634865 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82fb02b-9993-4740-bd23-8439427eebd0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:45.637576 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82fb02b-9993-4740-bd23-8439427eebd0 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:45.651477 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 445/907] 10.222.0.87 () {56 vars in 1251 bytes} [Tue Jun 21 18:13:45 2022] GET /identity/v3/endpoints?service_id=148a89a6d4284a62a366c6f25ab4837c&interface=public®ion_id=RegionOne => generated 194 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:45.660297 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82fb02b-9993-4740-bd23-8439427eebd0 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:45.779563 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12143ac0-80ef-409e-a775-9b60c9fd7a53 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:45.870648 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1777b88-8268-426f-bce9-d8b33a5be0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:45.870648 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1777b88-8268-426f-bce9-d8b33a5be0c1 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:45.870648 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1777b88-8268-426f-bce9-d8b33a5be0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:45.870648 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1777b88-8268-426f-bce9-d8b33a5be0c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:45.871318 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1777b88-8268-426f-bce9-d8b33a5be0c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:45.871318 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1777b88-8268-426f-bce9-d8b33a5be0c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:45.873304 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1777b88-8268-426f-bce9-d8b33a5be0c1 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:45.898523 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 463/908] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:13:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:45.901961 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82fb02b-9993-4740-bd23-8439427eebd0 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:45.908283 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1777b88-8268-426f-bce9-d8b33a5be0c1 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:45.968574 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bc993d-840b-4836-839e-2eada280da5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:45.969626 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc993d-840b-4836-839e-2eada280da5c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:45.970117 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc993d-840b-4836-839e-2eada280da5c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:45.970909 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc993d-840b-4836-839e-2eada280da5c None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:45.972056 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bc993d-840b-4836-839e-2eada280da5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:45.974566 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bc993d-840b-4836-839e-2eada280da5c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:45.986583 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 446/909] 10.222.0.87 () {56 vars in 1087 bytes} [Tue Jun 21 18:13:45 2022] GET /identity/v3/services => generated 1497 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:13:46.042781 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:46.137185 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1777b88-8268-426f-bce9-d8b33a5be0c1 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:46.214308 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfaffcb-ec67-42b9-b1f4-00e1e1d74d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:46.215861 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfaffcb-ec67-42b9-b1f4-00e1e1d74d92 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:46.216589 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfaffcb-ec67-42b9-b1f4-00e1e1d74d92 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:46.217092 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfaffcb-ec67-42b9-b1f4-00e1e1d74d92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:46.232583 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfaffcb-ec67-42b9-b1f4-00e1e1d74d92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:46.235865 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfaffcb-ec67-42b9-b1f4-00e1e1d74d92 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:46.266228 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-7dfaffcb-ec67-42b9-b1f4-00e1e1d74d92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:13:46.266228 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:46.266228 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:46.266228 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:46.266228 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:46.266228 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:46.266228 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:46.266228 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:46.266228 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:46.266228 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:46.266228 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:46.266228 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:46.266228 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:13:46.266228 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:13:46.266228 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:13:46.268218 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:13:46.268218 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:46.268218 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:46.268218 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:46.268218 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:46.268218 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:46.268218 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:46.268218 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:46.268218 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:46.268218 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:46.268218 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:46.268218 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:46.268218 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:46.268218 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:46.268218 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:46.268218 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:46.268218 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:46.270064 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:13:46.270064 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:13:46.270064 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:13:46.270064 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:13:46.270064 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:13:46.270064 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:13:46.270064 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:13:46.270064 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:46.270064 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 464/910] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:13:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:13:46.279471 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bc993d-840b-4836-839e-2eada280da5c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:46.324107 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:46.405236 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307261e1-1192-4a2d-a5df-1c69032d3027 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:46.405731 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307261e1-1192-4a2d-a5df-1c69032d3027 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:46.406227 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307261e1-1192-4a2d-a5df-1c69032d3027 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:46.409190 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 465/911] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:13:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:13:46.418523 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db545e1c-5c32-4008-9e44-ebe91bb6aa01 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:46.418992 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db545e1c-5c32-4008-9e44-ebe91bb6aa01 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:46.419469 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db545e1c-5c32-4008-9e44-ebe91bb6aa01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:46.450075 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-db545e1c-5c32-4008-9e44-ebe91bb6aa01 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:13:46.452637 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-db545e1c-5c32-4008-9e44-ebe91bb6aa01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:13:46.503269 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bc993d-840b-4836-839e-2eada280da5c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:46.580094 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5b7547-f186-42db-af5c-b984d68a7648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:46.581316 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b7547-f186-42db-af5c-b984d68a7648 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:46.582460 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b7547-f186-42db-af5c-b984d68a7648 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:46.582946 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b7547-f186-42db-af5c-b984d68a7648 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:46.585344 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5b7547-f186-42db-af5c-b984d68a7648 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:46.585963 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5b7547-f186-42db-af5c-b984d68a7648 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:46.589688 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5b7547-f186-42db-af5c-b984d68a7648 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:46.605480 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 466/912] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:13:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:13:46.641852 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 447/913] 10.222.0.87 () {56 vars in 1148 bytes} [Tue Jun 21 18:13:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:46.648625 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db545e1c-5c32-4008-9e44-ebe91bb6aa01 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:46.653046 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5b7547-f186-42db-af5c-b984d68a7648 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:46.774557 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:46.852664 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:46.898236 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5b7547-f186-42db-af5c-b984d68a7648 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:46.915409 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db545e1c-5c32-4008-9e44-ebe91bb6aa01 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:46.983850 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44af38-9657-44de-906f-e0c17be640a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:46.986983 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44af38-9657-44de-906f-e0c17be640a6 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:46.988044 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44af38-9657-44de-906f-e0c17be640a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:46.988752 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44af38-9657-44de-906f-e0c17be640a6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:46.991627 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c44af38-9657-44de-906f-e0c17be640a6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:46.994603 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c44af38-9657-44de-906f-e0c17be640a6 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:47.000632 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1d744c-a520-4291-a932-5d75eca0e13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:47.001985 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d744c-a520-4291-a932-5d75eca0e13e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:47.002393 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d744c-a520-4291-a932-5d75eca0e13e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:47.002928 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d744c-a520-4291-a932-5d75eca0e13e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:47.004445 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1d744c-a520-4291-a932-5d75eca0e13e None admin] RBAC: Adding query filter params (role.id=9fa867d83f3c406aabf51ddd8c7ed2aa, scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb, user.id=09c9fa7dccb8491c9ad3b43967345ea2) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:47.005052 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1d744c-a520-4291-a932-5d75eca0e13e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:47.007953 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1d744c-a520-4291-a932-5d75eca0e13e None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:47.018003 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-8c44af38-9657-44de-906f-e0c17be640a6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 21 18:13:47.018003 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:47.018003 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:13:47.018003 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:13:47.018003 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:13:47.018003 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:13:47.018003 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:13:47.018003 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:13:47.018003 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 21 18:13:47.018003 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:47.018003 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:13:47.018003 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:47.018003 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:47.018003 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:47.018003 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:47.018003 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:47.018003 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:47.020246 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:47.020246 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:47.020246 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:47.020246 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:47.020246 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:47.020246 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:47.020246 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:13:47.020246 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:13:47.020246 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:13:47.020246 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:13:47.020246 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:47.020246 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:47.020246 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:47.020246 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:47.020246 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:47.020246 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:47.028197 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:47.028197 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:47.028197 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:47.028197 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:47.028197 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:47.028197 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:47.028197 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:47.028197 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:47.028197 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:47.028197 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:47.028197 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:13:47.028197 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:13:47.028197 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 21 18:13:47.028197 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:47.028197 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 448/914] 10.222.0.87 () {56 vars in 1114 bytes} [Tue Jun 21 18:13:46 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:47.034574 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 467/915] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:13:46 2022] GET /identity/v3/role_assignments?user.id=09c9fa7dccb8491c9ad3b43967345ea2&scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb&role.id=9fa867d83f3c406aabf51ddd8c7ed2aa => generated 259 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:47.036209 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44af38-9657-44de-906f-e0c17be640a6 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:47.211132 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44af38-9657-44de-906f-e0c17be640a6 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:47.263579 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb25b6-ef51-470a-ac69-485f85b11778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:47.264768 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb25b6-ef51-470a-ac69-485f85b11778 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:47.265379 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb25b6-ef51-470a-ac69-485f85b11778 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:47.266157 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb25b6-ef51-470a-ac69-485f85b11778 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:47.267225 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bb25b6-ef51-470a-ac69-485f85b11778 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:47.267790 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bb25b6-ef51-470a-ac69-485f85b11778 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:47.269577 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bb25b6-ef51-470a-ac69-485f85b11778 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:47.281861 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 449/916] 10.222.0.87 () {56 vars in 1114 bytes} [Tue Jun 21 18:13:47 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:47.284729 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1d744c-a520-4291-a932-5d75eca0e13e None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:47.460559 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1d744c-a520-4291-a932-5d75eca0e13e None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:47.488621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:47.524544 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9cf4e3-174c-4b58-a09c-bd6ac0b9aba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:47.526044 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9cf4e3-174c-4b58-a09c-bd6ac0b9aba8 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:47.526696 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9cf4e3-174c-4b58-a09c-bd6ac0b9aba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:47.527273 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9cf4e3-174c-4b58-a09c-bd6ac0b9aba8 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:47.528503 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9cf4e3-174c-4b58-a09c-bd6ac0b9aba8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:47.529135 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9cf4e3-174c-4b58-a09c-bd6ac0b9aba8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:47.531320 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9cf4e3-174c-4b58-a09c-bd6ac0b9aba8 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:47.541227 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 468/917] 10.222.0.87 () {56 vars in 1114 bytes} [Tue Jun 21 18:13:47 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:13:47.548905 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb25b6-ef51-470a-ac69-485f85b11778 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:47.556373 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:47.721138 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb25b6-ef51-470a-ac69-485f85b11778 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:47.786206 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5255cedf-83ef-4a8c-93c9-7474d0c5313a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:47.788596 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255cedf-83ef-4a8c-93c9-7474d0c5313a None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:47.789370 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255cedf-83ef-4a8c-93c9-7474d0c5313a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:47.790133 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255cedf-83ef-4a8c-93c9-7474d0c5313a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:47.791933 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5255cedf-83ef-4a8c-93c9-7474d0c5313a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:47.794583 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5255cedf-83ef-4a8c-93c9-7474d0c5313a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:47.831507 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 450/918] 10.222.0.87 () {60 vars in 1144 bytes} [Tue Jun 21 18:13:47 2022] POST /identity/v3/endpoints => generated 346 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:13:48.925541 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec15c181-b36b-47e3-b1cd-f42410217bb4 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:48.926355 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec15c181-b36b-47e3-b1cd-f42410217bb4 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:48.926890 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec15c181-b36b-47e3-b1cd-f42410217bb4 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:48.929822 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 469/919] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:13:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:13:48.939086 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ec038-cf79-4557-8be6-1e59e7fc75ba None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:48.939881 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ec038-cf79-4557-8be6-1e59e7fc75ba None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:48.940048 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ec038-cf79-4557-8be6-1e59e7fc75ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:48.971529 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f00ec038-cf79-4557-8be6-1e59e7fc75ba None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:13:48.973690 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f00ec038-cf79-4557-8be6-1e59e7fc75ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:13:49.088598 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e5ddb-d2ec-4ee7-945e-9553016753c7 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:49.089027 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e5ddb-d2ec-4ee7-945e-9553016753c7 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:49.089539 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e5ddb-d2ec-4ee7-945e-9553016753c7 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:49.092785 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 470/920] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:13:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:13:49.093703 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:49.105190 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f5808-0088-4f4d-8abd-713119e77191 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:49.105730 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f5808-0088-4f4d-8abd-713119e77191 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:49.106231 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f5808-0088-4f4d-8abd-713119e77191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:49.124188 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 451/921] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:13:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:13:49.141836 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-cb1f5808-0088-4f4d-8abd-713119e77191 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:13:49.144489 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-cb1f5808-0088-4f4d-8abd-713119e77191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:13:49.166204 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00ec038-cf79-4557-8be6-1e59e7fc75ba None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:49.320353 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 471/922] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:13:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:13:49.360778 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f5808-0088-4f4d-8abd-713119e77191 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:49.404385 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00ec038-cf79-4557-8be6-1e59e7fc75ba None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:49.468358 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37bf6a-7338-49f9-badc-0b8c3bced8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:49.469708 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37bf6a-7338-49f9-badc-0b8c3bced8f3 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:49.470436 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37bf6a-7338-49f9-badc-0b8c3bced8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:49.470826 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37bf6a-7338-49f9-badc-0b8c3bced8f3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:49.471744 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e37bf6a-7338-49f9-badc-0b8c3bced8f3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:49.474206 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e37bf6a-7338-49f9-badc-0b8c3bced8f3 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:49.491814 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-0e37bf6a-7338-49f9-badc-0b8c3bced8f3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 18:13:49.491814 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:49.491814 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:13:49.491814 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:13:49.491814 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:13:49.491814 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:13:49.491814 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:13:49.491814 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:13:49.491814 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 18:13:49.491814 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:49.491814 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:13:49.491814 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:49.491814 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:49.491814 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:49.491814 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:49.491814 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:49.491814 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:49.493802 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:49.493802 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:49.493802 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:49.493802 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:49.493802 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:49.493802 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:49.493802 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:13:49.493802 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:13:49.493802 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:13:49.493802 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:13:49.493802 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:49.493802 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:49.493802 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:49.493802 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:49.493802 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:49.493802 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:49.495409 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:49.495409 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:49.495409 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:49.495409 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:49.495409 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:49.495409 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:49.495409 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:49.495409 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:49.495409 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:49.495409 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:49.495409 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:13:49.495409 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:13:49.495409 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 18:13:49.495409 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:49.495409 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 452/923] 10.222.0.87 () {56 vars in 1126 bytes} [Tue Jun 21 18:13:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:13:49.510173 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37bf6a-7338-49f9-badc-0b8c3bced8f3 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:49.558294 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f5808-0088-4f4d-8abd-713119e77191 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:49.576194 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:49.614943 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:49.623397 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c7a752-bcbb-4551-bb81-d29453a5b1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:49.624747 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c7a752-bcbb-4551-bb81-d29453a5b1ef None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:49.625322 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c7a752-bcbb-4551-bb81-d29453a5b1ef None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:49.635672 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c7a752-bcbb-4551-bb81-d29453a5b1ef None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:49.635672 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c7a752-bcbb-4551-bb81-d29453a5b1ef None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:49.635672 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c7a752-bcbb-4551-bb81-d29453a5b1ef None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:49.655827 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-87c7a752-bcbb-4551-bb81-d29453a5b1ef None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 21 18:13:49.655827 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:49.655827 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:13:49.655827 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:13:49.655827 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:13:49.655827 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:13:49.655827 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:13:49.655827 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:13:49.655827 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 21 18:13:49.655827 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:49.655827 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:13:49.655827 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:49.655827 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:49.655827 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:49.655827 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:49.655827 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:49.657908 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:49.657908 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:49.657908 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:49.657908 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:49.657908 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:49.657908 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:49.657908 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:49.657908 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:13:49.657908 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:13:49.657908 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:13:49.657908 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:13:49.657908 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:49.657908 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:49.657908 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:49.657908 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:49.657908 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:49.659046 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:49.659046 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:49.659046 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:49.659046 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:49.659046 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:49.659046 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:49.659046 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:49.659046 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:49.659046 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:49.659046 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:49.659046 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:49.659046 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:13:49.659046 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:13:49.659046 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 21 18:13:49.659046 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:49.659046 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 472/924] 10.222.0.87 () {56 vars in 1132 bytes} [Tue Jun 21 18:13:49 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:13:49.670525 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c7a752-bcbb-4551-bb81-d29453a5b1ef None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:49.743130 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37bf6a-7338-49f9-badc-0b8c3bced8f3 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:49.819701 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6912325-709b-4d87-a7ca-18573ef75304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:49.820806 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6912325-709b-4d87-a7ca-18573ef75304 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:49.821309 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6912325-709b-4d87-a7ca-18573ef75304 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:49.822195 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6912325-709b-4d87-a7ca-18573ef75304 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:49.823202 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6912325-709b-4d87-a7ca-18573ef75304 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:49.824389 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6912325-709b-4d87-a7ca-18573ef75304 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:49.826487 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6912325-709b-4d87-a7ca-18573ef75304 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:49.842899 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 453/925] 10.222.0.87 () {56 vars in 1122 bytes} [Tue Jun 21 18:13:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:49.854792 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6912325-709b-4d87-a7ca-18573ef75304 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:49.877801 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c7a752-bcbb-4551-bb81-d29453a5b1ef None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:49.951329 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5852b3-5a90-45a2-9192-7578f344aadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:49.953827 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5852b3-5a90-45a2-9192-7578f344aadf None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:49.954696 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5852b3-5a90-45a2-9192-7578f344aadf None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:49.955492 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5852b3-5a90-45a2-9192-7578f344aadf None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:49.957215 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5852b3-5a90-45a2-9192-7578f344aadf None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:49.958176 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5852b3-5a90-45a2-9192-7578f344aadf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:49.961401 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5852b3-5a90-45a2-9192-7578f344aadf None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:49.980864 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 473/926] 10.222.0.87 () {56 vars in 1126 bytes} [Tue Jun 21 18:13:49 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:49.983048 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5852b3-5a90-45a2-9192-7578f344aadf None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:50.032720 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6912325-709b-4d87-a7ca-18573ef75304 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:50.051638 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:50.063356 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:50.107814 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabef553-9d24-46b6-ba45-b7997f3d4531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:50.109014 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabef553-9d24-46b6-ba45-b7997f3d4531 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:50.109322 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabef553-9d24-46b6-ba45-b7997f3d4531 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:50.109730 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabef553-9d24-46b6-ba45-b7997f3d4531 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:50.110953 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabef553-9d24-46b6-ba45-b7997f3d4531 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:50.111333 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabef553-9d24-46b6-ba45-b7997f3d4531 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:50.113674 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabef553-9d24-46b6-ba45-b7997f3d4531 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:50.130011 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 454/927] 10.222.0.87 () {56 vars in 1122 bytes} [Tue Jun 21 18:13:49 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:50.133335 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabef553-9d24-46b6-ba45-b7997f3d4531 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:50.227600 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5852b3-5a90-45a2-9192-7578f344aadf None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:50.301211 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e506a-cf9d-4cb6-8947-dedc108968f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:50.303025 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e506a-cf9d-4cb6-8947-dedc108968f3 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:50.303873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e506a-cf9d-4cb6-8947-dedc108968f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:50.304648 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e506a-cf9d-4cb6-8947-dedc108968f3 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:50.306379 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e506a-cf9d-4cb6-8947-dedc108968f3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:50.307251 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e506a-cf9d-4cb6-8947-dedc108968f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:50.310434 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e506a-cf9d-4cb6-8947-dedc108968f3 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:50.328753 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 474/928] 10.222.0.87 () {56 vars in 1126 bytes} [Tue Jun 21 18:13:49 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:50.331262 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e506a-cf9d-4cb6-8947-dedc108968f3 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:50.366045 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabef553-9d24-46b6-ba45-b7997f3d4531 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:50.444002 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15976661-3eba-4cd3-af6f-2c41f12fe952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:50.445314 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15976661-3eba-4cd3-af6f-2c41f12fe952 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:50.445842 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15976661-3eba-4cd3-af6f-2c41f12fe952 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:50.446273 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15976661-3eba-4cd3-af6f-2c41f12fe952 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:50.447814 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15976661-3eba-4cd3-af6f-2c41f12fe952 None admin] RBAC: Adding query filter params (interface=public, service_id=cb8fc51a0dfa4523b6309034c0c5cadd, region_id=RegionOne) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:50.448365 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15976661-3eba-4cd3-af6f-2c41f12fe952 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:50.451422 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15976661-3eba-4cd3-af6f-2c41f12fe952 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:50.463033 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 455/929] 10.222.0.87 () {56 vars in 1251 bytes} [Tue Jun 21 18:13:50 2022] GET /identity/v3/endpoints?service_id=cb8fc51a0dfa4523b6309034c0c5cadd&interface=public®ion_id=RegionOne => generated 194 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:50.466028 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c65ce0-71bb-4885-b9c6-214da88cd3e1 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:50.466988 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c65ce0-71bb-4885-b9c6-214da88cd3e1 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:50.467766 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c65ce0-71bb-4885-b9c6-214da88cd3e1 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:50.471145 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 456/930] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:13:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:13:50.474402 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c65ce0-71bb-4885-b9c6-214da88cd3e1 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:50.564352 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:50.569031 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e506a-cf9d-4cb6-8947-dedc108968f3 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:50.576442 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:50.640880 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8028be04-f5ba-4299-90e6-5ba755ee6612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:50.642040 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8028be04-f5ba-4299-90e6-5ba755ee6612 None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:50.642520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8028be04-f5ba-4299-90e6-5ba755ee6612 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:50.643050 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8028be04-f5ba-4299-90e6-5ba755ee6612 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:50.644592 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8028be04-f5ba-4299-90e6-5ba755ee6612 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:50.647597 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8028be04-f5ba-4299-90e6-5ba755ee6612 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:50.686642 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 475/931] 10.222.0.87 () {60 vars in 1144 bytes} [Tue Jun 21 18:13:50 2022] POST /identity/v3/endpoints => generated 347 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:13:50.689441 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a317558-255e-4df5-9c14-9802fa028c46 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:50.690078 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a317558-255e-4df5-9c14-9802fa028c46 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:50.690727 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a317558-255e-4df5-9c14-9802fa028c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:50.726253 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-4a317558-255e-4df5-9c14-9802fa028c46 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:13:50.728278 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4a317558-255e-4df5-9c14-9802fa028c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:13:50.733578 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c65ce0-71bb-4885-b9c6-214da88cd3e1 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:50.824077 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e304c205-3430-47e0-897b-d6491bb061eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:50.825150 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e304c205-3430-47e0-897b-d6491bb061eb None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:50.825691 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e304c205-3430-47e0-897b-d6491bb061eb None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:50.826052 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e304c205-3430-47e0-897b-d6491bb061eb None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:50.827295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e304c205-3430-47e0-897b-d6491bb061eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:50.829250 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e304c205-3430-47e0-897b-d6491bb061eb None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:50.846269 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 457/932] 10.222.0.87 () {56 vars in 1087 bytes} [Tue Jun 21 18:13:50 2022] GET /identity/v3/services => generated 1497 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:13:50.913794 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 476/933] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:13:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:13:50.954653 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e304c205-3430-47e0-897b-d6491bb061eb None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:51.225530 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e304c205-3430-47e0-897b-d6491bb061eb None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:51.231006 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:51.294861 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b067368e-8f6e-4043-b3f5-589ec53dd396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:51.296587 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067368e-8f6e-4043-b3f5-589ec53dd396 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:51.297334 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067368e-8f6e-4043-b3f5-589ec53dd396 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:51.298216 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067368e-8f6e-4043-b3f5-589ec53dd396 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:51.315910 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b067368e-8f6e-4043-b3f5-589ec53dd396 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:13:51.329858 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-b067368e-8f6e-4043-b3f5-589ec53dd396 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 18:13:51.329858 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:51.329858 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:51.329858 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:51.329858 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:51.329858 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:51.329858 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:51.329858 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:51.329858 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:51.329858 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:51.329858 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:51.329858 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:51.329858 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:13:51.329858 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:13:51.329858 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:13:51.329858 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:13:51.331328 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:13:51.331328 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:13:51.331328 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:13:51.331328 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:13:51.331328 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:51.331328 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:51.331328 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:51.331328 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:51.331328 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:51.331328 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:51.331328 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:51.331328 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:51.331328 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:51.331328 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:51.331328 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:51.331328 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:51.332684 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:51.332684 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:51.332684 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:51.332684 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:51.332684 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:13:51.332684 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:13:51.332684 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:13:51.332684 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:13:51.332684 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:13:51.332684 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:13:51.332684 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 18:13:51.332684 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:51.336345 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 458/934] 10.222.0.87 () {56 vars in 1120 bytes} [Tue Jun 21 18:13:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:51.348593 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a317558-255e-4df5-9c14-9802fa028c46 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:51.477408 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af8813-1100-41b0-a238-02aae381f4be None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:51.477886 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af8813-1100-41b0-a238-02aae381f4be None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:51.478370 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af8813-1100-41b0-a238-02aae381f4be None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:51.485803 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 459/935] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:13:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:13:51.490856 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe9701-d052-403e-b875-4f7d68091656 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:51.491404 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe9701-d052-403e-b875-4f7d68091656 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:51.491823 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe9701-d052-403e-b875-4f7d68091656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:51.529945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-cffe9701-d052-403e-b875-4f7d68091656 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:13:51.532158 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-cffe9701-d052-403e-b875-4f7d68091656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:13:51.575280 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:51.617473 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a317558-255e-4df5-9c14-9802fa028c46 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:51.701984 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d909e6-059c-4a53-87ca-86f9bcffbb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:51.703043 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d909e6-059c-4a53-87ca-86f9bcffbb98 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:51.703490 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d909e6-059c-4a53-87ca-86f9bcffbb98 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:51.704050 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d909e6-059c-4a53-87ca-86f9bcffbb98 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:51.705788 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d909e6-059c-4a53-87ca-86f9bcffbb98 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:51.709711 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d909e6-059c-4a53-87ca-86f9bcffbb98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:51.709967 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 460/936] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:13:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:13:51.716085 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d909e6-059c-4a53-87ca-86f9bcffbb98 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:51.737619 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 477/937] 10.222.0.87 () {56 vars in 1115 bytes} [Tue Jun 21 18:13:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:51.745892 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe9701-d052-403e-b875-4f7d68091656 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:51.748649 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d909e6-059c-4a53-87ca-86f9bcffbb98 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:51.902112 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:51.914577 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe9701-d052-403e-b875-4f7d68091656 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:51.967546 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d909e6-059c-4a53-87ca-86f9bcffbb98 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:51.994113 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d38d9c6-e81a-4eca-9834-7a5d3d67f199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:52.003626 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d38d9c6-e81a-4eca-9834-7a5d3d67f199 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:52.006852 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d38d9c6-e81a-4eca-9834-7a5d3d67f199 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:52.009882 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d38d9c6-e81a-4eca-9834-7a5d3d67f199 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:52.036252 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d38d9c6-e81a-4eca-9834-7a5d3d67f199 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:52.039499 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d38d9c6-e81a-4eca-9834-7a5d3d67f199 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:52.046335 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0254e6-1570-4b92-98cf-decab3aa6baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:52.047915 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0254e6-1570-4b92-98cf-decab3aa6baa None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:52.048625 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0254e6-1570-4b92-98cf-decab3aa6baa None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:52.049316 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0254e6-1570-4b92-98cf-decab3aa6baa None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:52.050908 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0254e6-1570-4b92-98cf-decab3aa6baa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:52.054966 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0254e6-1570-4b92-98cf-decab3aa6baa None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:52.057955 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: WARNING keystone.server.flask.common [None req-0d0254e6-1570-4b92-98cf-decab3aa6baa None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 21 18:13:52.074148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-9d38d9c6-e81a-4eca-9834-7a5d3d67f199 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:52.074148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:52.074148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:52.074148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:52.074148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:52.074148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:52.074148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:52.074148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:52.074148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:52.074148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:52.074148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:52.074148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:52.074148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:13:52.074148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:13:52.074148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:13:52.077270 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:13:52.077270 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:52.077270 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:52.077270 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:52.077270 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:52.077270 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:52.077270 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:52.077270 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:52.077270 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:52.077270 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:52.077270 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:52.077270 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:52.077270 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:52.077270 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:52.077270 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:52.077270 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:52.077270 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:52.079260 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:13:52.079260 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:13:52.079260 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:52.079260 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:52.083918 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 461/938] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:13:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:52.086358 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d38d9c6-e81a-4eca-9834-7a5d3d67f199 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:52.109387 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.sql.core [None req-0d0254e6-1570-4b92-98cf-decab3aa6baa None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 21 18:13:52.109387 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 21 18:13:52.109387 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [parameters: {'id': 'a3b92e395a6a48219f04c31b87bc5fb0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 21 18:13:52.109387 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80400) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 21 18:13:52.148251 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-0d0254e6-1570-4b92-98cf-decab3aa6baa None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a3b92e395a6a48219f04c31b87bc5fb0.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a3b92e395a6a48219f04c31b87bc5fb0. Jun 21 18:13:52.148251 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:52.148251 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 21 18:13:52.148251 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 21 18:13:52.148251 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 21 18:13:52.148251 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 21 18:13:52.148251 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 21 18:13:52.148251 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application result = self._query(query) Jun 21 18:13:52.148251 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 21 18:13:52.148251 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application conn.query(q) Jun 21 18:13:52.148251 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 21 18:13:52.148251 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 21 18:13:52.148251 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 21 18:13:52.149968 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application result.read() Jun 21 18:13:52.149968 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 21 18:13:52.149968 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 21 18:13:52.149968 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 21 18:13:52.149968 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 21 18:13:52.149968 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 21 18:13:52.149968 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 21 18:13:52.149968 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 21 18:13:52.149968 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 21 18:13:52.149968 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 21 18:13:52.149968 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:52.149968 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 21 18:13:52.149968 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:52.149968 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:52.149968 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 21 18:13:52.149968 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 21 18:13:52.149968 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 21 18:13:52.149968 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 21 18:13:52.152473 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 21 18:13:52.152473 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application next(self.gen) Jun 21 18:13:52.152473 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 21 18:13:52.152473 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application yield resource Jun 21 18:13:52.152473 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 21 18:13:52.152473 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application next(self.gen) Jun 21 18:13:52.152473 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 21 18:13:52.152473 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self.session.rollback() Jun 21 18:13:52.152473 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 21 18:13:52.152473 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self.force_reraise() Jun 21 18:13:52.152473 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 21 18:13:52.152473 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise self.value Jun 21 18:13:52.152473 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 21 18:13:52.152473 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 21 18:13:52.152473 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 21 18:13:52.152473 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application session.commit() Jun 21 18:13:52.152473 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 21 18:13:52.154309 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 21 18:13:52.154309 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 21 18:13:52.154309 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._prepare_impl() Jun 21 18:13:52.154309 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 21 18:13:52.154309 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self.session.flush() Jun 21 18:13:52.154309 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 21 18:13:52.154309 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._flush(objects) Jun 21 18:13:52.154309 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 21 18:13:52.154309 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 21 18:13:52.154309 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 21 18:13:52.154309 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application compat.raise_( Jun 21 18:13:52.154309 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 21 18:13:52.154309 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception Jun 21 18:13:52.154309 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 21 18:13:52.154309 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application flush_context.execute() Jun 21 18:13:52.154309 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 21 18:13:52.154309 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rec.execute(self) Jun 21 18:13:52.156059 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 21 18:13:52.156059 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 21 18:13:52.156059 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 21 18:13:52.156059 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 21 18:13:52.156059 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 21 18:13:52.156059 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 21 18:13:52.156059 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 21 18:13:52.156059 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 21 18:13:52.156059 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 21 18:13:52.156059 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 21 18:13:52.156059 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 21 18:13:52.156059 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 21 18:13:52.156059 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 21 18:13:52.156059 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 21 18:13:52.156059 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 21 18:13:52.161431 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 21 18:13:52.161431 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 21 18:13:52.161431 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception Jun 21 18:13:52.161431 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 21 18:13:52.161431 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 21 18:13:52.161431 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 21 18:13:52.161431 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 21 18:13:52.161431 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 21 18:13:52.161431 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application result = self._query(query) Jun 21 18:13:52.161431 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 21 18:13:52.161431 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application conn.query(q) Jun 21 18:13:52.161431 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 21 18:13:52.161431 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 21 18:13:52.161431 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 21 18:13:52.161431 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application result.read() Jun 21 18:13:52.161431 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 21 18:13:52.161431 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 21 18:13:52.164860 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 21 18:13:52.164860 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 21 18:13:52.164860 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 21 18:13:52.164860 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 21 18:13:52.164860 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 21 18:13:52.164860 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 21 18:13:52.164860 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 21 18:13:52.164860 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 21 18:13:52.164860 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [parameters: {'id': 'a3b92e395a6a48219f04c31b87bc5fb0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 21 18:13:52.164860 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 21 18:13:52.164860 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:52.164860 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:13:52.164860 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:52.164860 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:52.164860 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 21 18:13:52.169241 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 21 18:13:52.169241 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 21 18:13:52.169241 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 21 18:13:52.169241 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 21 18:13:52.169241 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:52.169241 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:13:52.169241 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:52.169241 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:52.169241 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:52.169241 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:52.169241 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:52.169241 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:52.169241 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:52.169241 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:52.169241 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:52.169241 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:52.169241 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:52.180245 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:52.180245 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 21 18:13:52.180245 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 21 18:13:52.180245 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:52.180245 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:52.180245 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 21 18:13:52.180245 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 21 18:13:52.180245 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a3b92e395a6a48219f04c31b87bc5fb0. Jun 21 18:13:52.180245 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:52.180245 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 478/939] 10.222.0.87 () {60 vars in 1140 bytes} [Tue Jun 21 18:13:51 2022] POST /identity/v3/projects => generated 262 bytes in 424 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:52.192875 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0254e6-1570-4b92-98cf-decab3aa6baa None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:52.201238 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:52.299926 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d38d9c6-e81a-4eca-9834-7a5d3d67f199 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:52.370145 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a982a4-8673-41e7-b0c9-572a487d4646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:52.371400 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a982a4-8673-41e7-b0c9-572a487d4646 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:52.371939 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a982a4-8673-41e7-b0c9-572a487d4646 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:52.372463 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a982a4-8673-41e7-b0c9-572a487d4646 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:52.373954 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a982a4-8673-41e7-b0c9-572a487d4646 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:52.374512 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a982a4-8673-41e7-b0c9-572a487d4646 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:52.377310 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a982a4-8673-41e7-b0c9-572a487d4646 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:52.419933 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 462/940] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:13:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:52.422092 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a982a4-8673-41e7-b0c9-572a487d4646 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:52.426645 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:52.452096 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0254e6-1570-4b92-98cf-decab3aa6baa None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:52.526319 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b73e3b-dfb5-4c7e-8836-77a66ed0de8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:52.528170 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b73e3b-dfb5-4c7e-8836-77a66ed0de8f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:52.528895 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b73e3b-dfb5-4c7e-8836-77a66ed0de8f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:52.529774 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b73e3b-dfb5-4c7e-8836-77a66ed0de8f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:52.544102 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b73e3b-dfb5-4c7e-8836-77a66ed0de8f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:52.547468 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b73e3b-dfb5-4c7e-8836-77a66ed0de8f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:52.578330 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-34b73e3b-dfb5-4c7e-8836-77a66ed0de8f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:13:52.578330 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:52.578330 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:52.578330 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:52.578330 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:52.578330 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:52.578330 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:52.578330 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:52.578330 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:52.578330 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:52.578330 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:52.578330 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:52.578330 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:13:52.578330 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:13:52.578330 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:13:52.580324 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:13:52.580324 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:52.580324 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:52.580324 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:52.580324 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:52.580324 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:52.580324 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:52.580324 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:52.580324 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:52.580324 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:52.580324 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:52.580324 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:52.580324 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:52.580324 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:52.580324 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:52.580324 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:52.580324 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:52.582297 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:13:52.582297 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:13:52.582297 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:13:52.582297 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:13:52.582297 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:13:52.582297 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:13:52.582297 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:13:52.582297 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:52.593520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 479/941] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:13:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 395 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:52.595859 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b73e3b-dfb5-4c7e-8836-77a66ed0de8f None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:52.599641 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a982a4-8673-41e7-b0c9-572a487d4646 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:52.660215 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff0e04c-c811-4c91-bfa8-bc973046aa3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:52.662075 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0e04c-c811-4c91-bfa8-bc973046aa3d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:52.662758 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0e04c-c811-4c91-bfa8-bc973046aa3d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:52.663384 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0e04c-c811-4c91-bfa8-bc973046aa3d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:52.679805 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff0e04c-c811-4c91-bfa8-bc973046aa3d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:52.683079 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff0e04c-c811-4c91-bfa8-bc973046aa3d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:52.716768 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-aff0e04c-c811-4c91-bfa8-bc973046aa3d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 18:13:52.716768 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:52.716768 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:52.716768 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:52.716768 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:52.716768 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:52.716768 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:52.716768 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:52.716768 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:52.716768 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:52.716768 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:52.716768 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:52.716768 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:13:52.716768 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:13:52.716768 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:13:52.716768 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:13:52.719142 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:52.719142 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:52.719142 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:13:52.719142 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:13:52.719142 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:13:52.719142 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:13:52.719142 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:52.719142 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:52.719142 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:52.719142 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:52.719142 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:52.719142 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:52.719142 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:52.719142 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:52.719142 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:52.719142 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:52.719142 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:52.719142 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:52.721582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:52.721582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:52.721582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:13:52.721582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:13:52.721582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:13:52.721582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:13:52.721582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:13:52.721582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:13:52.721582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 18:13:52.721582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:52.729863 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 463/942] 10.222.0.87 () {56 vars in 1117 bytes} [Tue Jun 21 18:13:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:52.734899 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff0e04c-c811-4c91-bfa8-bc973046aa3d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:52.743885 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:52.835057 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b73e3b-dfb5-4c7e-8836-77a66ed0de8f None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:52.877208 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:52.920394 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0836f233-629a-4c2f-acee-2bda4ab94aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:52.921954 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836f233-629a-4c2f-acee-2bda4ab94aba None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:52.922658 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836f233-629a-4c2f-acee-2bda4ab94aba None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:52.923458 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836f233-629a-4c2f-acee-2bda4ab94aba None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:52.925187 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0836f233-629a-4c2f-acee-2bda4ab94aba None admin] RBAC: Adding query filter params (name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:52.926018 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0836f233-629a-4c2f-acee-2bda4ab94aba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:52.929054 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0836f233-629a-4c2f-acee-2bda4ab94aba None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:52.962939 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff0e04c-c811-4c91-bfa8-bc973046aa3d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:52.981367 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 480/943] 10.222.0.87 () {56 vars in 1112 bytes} [Tue Jun 21 18:13:52 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:53.041253 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16791a49-a76b-4de2-b9d1-1d6f856f2483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:53.042551 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16791a49-a76b-4de2-b9d1-1d6f856f2483 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:53.043051 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16791a49-a76b-4de2-b9d1-1d6f856f2483 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:53.043549 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16791a49-a76b-4de2-b9d1-1d6f856f2483 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:53.045230 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16791a49-a76b-4de2-b9d1-1d6f856f2483 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:53.045829 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16791a49-a76b-4de2-b9d1-1d6f856f2483 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:53.048698 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16791a49-a76b-4de2-b9d1-1d6f856f2483 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:53.093968 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 464/944] 10.222.0.87 () {56 vars in 1149 bytes} [Tue Jun 21 18:13:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:13:53.098347 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0836f233-629a-4c2f-acee-2bda4ab94aba None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:53.334251 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0836f233-629a-4c2f-acee-2bda4ab94aba None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:53.378213 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:53.400222 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bf4f67-8676-4a3b-b44b-b20213da8029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:53.401686 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf4f67-8676-4a3b-b44b-b20213da8029 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:53.402217 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf4f67-8676-4a3b-b44b-b20213da8029 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:53.402711 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf4f67-8676-4a3b-b44b-b20213da8029 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:53.428643 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bf4f67-8676-4a3b-b44b-b20213da8029 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:53.432712 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bf4f67-8676-4a3b-b44b-b20213da8029 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:53.478623 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-29bf4f67-8676-4a3b-b44b-b20213da8029 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:53.478623 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:53.478623 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:53.478623 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:53.478623 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:53.478623 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:53.478623 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:53.478623 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:53.478623 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:53.478623 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:53.478623 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:53.478623 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:53.478623 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:13:53.478623 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:13:53.478623 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:13:53.481550 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:13:53.481550 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:53.481550 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:53.481550 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:53.481550 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:53.481550 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:53.481550 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:53.481550 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:53.481550 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:53.481550 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:53.481550 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:53.481550 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:53.481550 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:53.481550 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:53.481550 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:53.481550 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:53.481550 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:53.483583 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:13:53.483583 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:13:53.483583 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:53.483583 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:53.489854 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 481/945] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:13:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:53.492935 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16791a49-a76b-4de2-b9d1-1d6f856f2483 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:53.729210 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16791a49-a76b-4de2-b9d1-1d6f856f2483 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:53.794210 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:53.813316 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea2fd14-eeaf-4ae8-a4be-fca42dee1f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:53.813316 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2fd14-eeaf-4ae8-a4be-fca42dee1f04 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:53.813316 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2fd14-eeaf-4ae8-a4be-fca42dee1f04 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:53.813316 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2fd14-eeaf-4ae8-a4be-fca42dee1f04 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:53.814042 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea2fd14-eeaf-4ae8-a4be-fca42dee1f04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:53.814042 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea2fd14-eeaf-4ae8-a4be-fca42dee1f04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:53.817280 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea2fd14-eeaf-4ae8-a4be-fca42dee1f04 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:53.842304 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 465/946] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:13:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:53.857722 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bf4f67-8676-4a3b-b44b-b20213da8029 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:54.084812 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bf4f67-8676-4a3b-b44b-b20213da8029 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:54.161128 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c749749-9c15-45a5-8e5b-58f4c6bddda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:54.162631 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c749749-9c15-45a5-8e5b-58f4c6bddda3 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:54.163114 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c749749-9c15-45a5-8e5b-58f4c6bddda3 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:54.163600 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c749749-9c15-45a5-8e5b-58f4c6bddda3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:54.179439 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c749749-9c15-45a5-8e5b-58f4c6bddda3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:54.182507 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c749749-9c15-45a5-8e5b-58f4c6bddda3 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:54.212346 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-4c749749-9c15-45a5-8e5b-58f4c6bddda3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:13:54.212346 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:54.212346 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:54.212346 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:54.212346 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:54.212346 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:54.212346 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:54.212346 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:54.212346 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:54.212346 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:54.212346 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:54.212346 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:54.212346 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:13:54.212346 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:13:54.212346 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:13:54.213595 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:13:54.213595 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:54.213595 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:54.213595 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:54.213595 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:54.213595 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:54.213595 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:54.213595 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:54.213595 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:54.213595 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:54.213595 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:54.213595 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:54.213595 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:54.213595 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:54.213595 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:54.213595 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:54.213595 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:54.214823 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:13:54.214823 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:13:54.214823 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:13:54.214823 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:13:54.214823 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:13:54.214823 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:13:54.214823 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:13:54.214823 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:54.215917 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 482/947] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:13:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:54.226626 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea2fd14-eeaf-4ae8-a4be-fca42dee1f04 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:54.376896 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9c4b6-d6cb-4caa-97b4-cba3c9536c31 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:54.377192 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9c4b6-d6cb-4caa-97b4-cba3c9536c31 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:54.377674 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9c4b6-d6cb-4caa-97b4-cba3c9536c31 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:54.387966 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 483/948] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:13:54 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:13:54.390426 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626fb978-de6c-4797-a222-423c3a19ce41 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:54.390936 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626fb978-de6c-4797-a222-423c3a19ce41 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:54.391345 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626fb978-de6c-4797-a222-423c3a19ce41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:54.428025 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-626fb978-de6c-4797-a222-423c3a19ce41 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:13:54.430988 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-626fb978-de6c-4797-a222-423c3a19ce41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:13:54.448319 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea2fd14-eeaf-4ae8-a4be-fca42dee1f04 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:54.498120 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:54.534222 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b0220-0ea6-4585-8b3d-4252e4951c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:54.535506 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b0220-0ea6-4585-8b3d-4252e4951c43 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:54.536812 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b0220-0ea6-4585-8b3d-4252e4951c43 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:54.536812 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b0220-0ea6-4585-8b3d-4252e4951c43 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:54.538458 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31b0220-0ea6-4585-8b3d-4252e4951c43 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:54.539326 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31b0220-0ea6-4585-8b3d-4252e4951c43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:54.542162 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31b0220-0ea6-4585-8b3d-4252e4951c43 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:54.573729 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 466/949] 10.222.0.87 () {56 vars in 1148 bytes} [Tue Jun 21 18:13:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:54.573729 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa996a-3e08-434e-b800-87a0491d8ec9 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:54.573729 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa996a-3e08-434e-b800-87a0491d8ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:54.574300 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa996a-3e08-434e-b800-87a0491d8ec9 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:54.576471 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 467/950] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:13:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:13:54.582183 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfa996a-3e08-434e-b800-87a0491d8ec9 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:54.603660 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 484/951] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:13:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:13:54.609259 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ac224-762a-4c8a-bc3d-cda5e9331072 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:54.609259 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ac224-762a-4c8a-bc3d-cda5e9331072 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:54.609259 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ac224-762a-4c8a-bc3d-cda5e9331072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:54.637536 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-d22ac224-762a-4c8a-bc3d-cda5e9331072 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:13:54.639632 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d22ac224-762a-4c8a-bc3d-cda5e9331072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:13:54.682270 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:54.801338 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 485/952] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:13:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:13:54.804310 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22ac224-762a-4c8a-bc3d-cda5e9331072 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:54.828219 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfa996a-3e08-434e-b800-87a0491d8ec9 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:54.891202 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b11fd-cd34-4357-b33f-89979600dbc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:54.893608 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b11fd-cd34-4357-b33f-89979600dbc3 None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:54.894250 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b11fd-cd34-4357-b33f-89979600dbc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:54.894545 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b11fd-cd34-4357-b33f-89979600dbc3 None admin] PATH_INFO: `/v3/projects/5f30c3ddb5e44e9299ff228fe94c27fb/users/09c9fa7dccb8491c9ad3b43967345ea2/roles/9fa867d83f3c406aabf51ddd8c7ed2aa` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:54.902759 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b11fd-cd34-4357-b33f-89979600dbc3 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f30c3ddb5e44e9299ff228fe94c27fb, user_id=09c9fa7dccb8491c9ad3b43967345ea2, role_id=9fa867d83f3c406aabf51ddd8c7ed2aa)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:54.904905 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b11fd-cd34-4357-b33f-89979600dbc3 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:54.928529 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 468/953] 10.222.0.87 () {58 vars in 1439 bytes} [Tue Jun 21 18:13:54 2022] PUT /identity/v3/projects/5f30c3ddb5e44e9299ff228fe94c27fb/users/09c9fa7dccb8491c9ad3b43967345ea2/roles/9fa867d83f3c406aabf51ddd8c7ed2aa => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:13:54.935293 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b11fd-cd34-4357-b33f-89979600dbc3 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:55.048731 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:55.062680 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22ac224-762a-4c8a-bc3d-cda5e9331072 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:55.106895 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b11fd-cd34-4357-b33f-89979600dbc3 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:55.109557 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:55.145929 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4253edc-32d1-482b-9074-a45ba570e767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:55.146736 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4253edc-32d1-482b-9074-a45ba570e767 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:55.147245 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4253edc-32d1-482b-9074-a45ba570e767 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:55.148176 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4253edc-32d1-482b-9074-a45ba570e767 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:55.149237 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4253edc-32d1-482b-9074-a45ba570e767 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:55.152094 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4253edc-32d1-482b-9074-a45ba570e767 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:55.164541 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a555cd-02ed-4d02-abbb-6e0a21dea811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:55.166000 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a555cd-02ed-4d02-abbb-6e0a21dea811 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:55.166615 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a555cd-02ed-4d02-abbb-6e0a21dea811 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:55.167162 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a555cd-02ed-4d02-abbb-6e0a21dea811 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:55.169183 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a555cd-02ed-4d02-abbb-6e0a21dea811 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:55.171183 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a555cd-02ed-4d02-abbb-6e0a21dea811 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:55.179247 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-c4253edc-32d1-482b-9074-a45ba570e767 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 18:13:55.179247 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:55.179247 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:13:55.179247 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:13:55.179247 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:13:55.179247 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:13:55.179247 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:13:55.179247 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:13:55.179247 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 18:13:55.179247 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:55.179247 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:13:55.179247 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:55.179247 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:55.179247 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:55.179247 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:55.179247 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:55.179247 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:55.180717 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:55.180717 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:55.180717 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:55.180717 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:55.180717 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:55.180717 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:55.180717 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:13:55.180717 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:13:55.180717 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:13:55.180717 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:13:55.180717 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:55.180717 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:55.180717 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:55.180717 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:55.180717 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:55.180717 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:55.181664 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:55.181664 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:55.181664 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:55.181664 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:55.181664 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:55.181664 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:55.181664 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:55.181664 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:55.181664 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:55.181664 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:55.181664 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:13:55.181664 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:13:55.181664 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 18:13:55.181664 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:55.182594 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 486/954] 10.222.0.87 () {56 vars in 1126 bytes} [Tue Jun 21 18:13:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:55.191791 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-98a555cd-02ed-4d02-abbb-6e0a21dea811 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 21 18:13:55.191791 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:55.191791 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:13:55.191791 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:13:55.191791 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:13:55.191791 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:13:55.191791 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:13:55.191791 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:13:55.191791 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 21 18:13:55.191791 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:55.191791 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:13:55.191791 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:55.191791 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:55.191791 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:55.191791 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:55.191791 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:55.191791 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:55.193195 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:55.193195 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:55.193195 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:55.193195 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:55.193195 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:55.193195 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:55.193195 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:13:55.193195 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:13:55.193195 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:13:55.193195 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:13:55.193195 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:55.193195 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:55.193195 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:55.193195 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:55.193195 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:55.193195 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:55.194216 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:55.194216 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:55.194216 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:55.194216 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:55.194216 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:55.194216 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:55.194216 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:55.194216 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:55.194216 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:55.194216 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:55.194216 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:13:55.194216 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:13:55.194216 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 21 18:13:55.194216 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:55.195031 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4253edc-32d1-482b-9074-a45ba570e767 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:55.197876 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 469/955] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:13:54 2022] GET /identity/v3/services/compute => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:55.210142 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a555cd-02ed-4d02-abbb-6e0a21dea811 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:55.405516 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a555cd-02ed-4d02-abbb-6e0a21dea811 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:55.437414 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4253edc-32d1-482b-9074-a45ba570e767 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:55.491700 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331a8dfe-9b71-40fa-a2a5-feadb6439b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:55.493665 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a8dfe-9b71-40fa-a2a5-feadb6439b19 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:55.494153 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a8dfe-9b71-40fa-a2a5-feadb6439b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:55.494904 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a8dfe-9b71-40fa-a2a5-feadb6439b19 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:55.497050 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331a8dfe-9b71-40fa-a2a5-feadb6439b19 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:55.497787 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331a8dfe-9b71-40fa-a2a5-feadb6439b19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:55.501143 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331a8dfe-9b71-40fa-a2a5-feadb6439b19 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:55.512389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99581f3-203d-4e61-9390-7a821e3bce00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:55.513495 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99581f3-203d-4e61-9390-7a821e3bce00 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:55.513760 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 470/956] 10.222.0.87 () {56 vars in 1112 bytes} [Tue Jun 21 18:13:55 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:55.514046 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99581f3-203d-4e61-9390-7a821e3bce00 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:55.514569 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99581f3-203d-4e61-9390-7a821e3bce00 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:55.515558 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99581f3-203d-4e61-9390-7a821e3bce00 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:55.516073 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99581f3-203d-4e61-9390-7a821e3bce00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:55.521827 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99581f3-203d-4e61-9390-7a821e3bce00 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:55.524141 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331a8dfe-9b71-40fa-a2a5-feadb6439b19 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:55.530424 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 487/957] 10.222.0.87 () {56 vars in 1122 bytes} [Tue Jun 21 18:13:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:13:55.547005 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99581f3-203d-4e61-9390-7a821e3bce00 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:55.598786 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:55.636698 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:55.771588 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331a8dfe-9b71-40fa-a2a5-feadb6439b19 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:55.821181 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99581f3-203d-4e61-9390-7a821e3bce00 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:55.853867 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede0a8d-14b9-478e-9bcd-770321f0a5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:55.855045 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede0a8d-14b9-478e-9bcd-770321f0a5af None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:55.855727 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede0a8d-14b9-478e-9bcd-770321f0a5af None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:55.856570 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede0a8d-14b9-478e-9bcd-770321f0a5af None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:55.857870 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ede0a8d-14b9-478e-9bcd-770321f0a5af None admin] RBAC: Adding query filter params (type=compute) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:55.858676 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ede0a8d-14b9-478e-9bcd-770321f0a5af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:55.861020 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ede0a8d-14b9-478e-9bcd-770321f0a5af None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:55.874275 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 471/958] 10.222.0.87 () {56 vars in 1112 bytes} [Tue Jun 21 18:13:55 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:55.905092 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc995a-71aa-4976-b83e-e8e264b50d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:55.906327 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc995a-71aa-4976-b83e-e8e264b50d7c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:55.906814 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc995a-71aa-4976-b83e-e8e264b50d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:55.907274 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc995a-71aa-4976-b83e-e8e264b50d7c None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:55.908654 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc995a-71aa-4976-b83e-e8e264b50d7c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:55.909183 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc995a-71aa-4976-b83e-e8e264b50d7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:55.912539 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc995a-71aa-4976-b83e-e8e264b50d7c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:55.923545 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 488/959] 10.222.0.87 () {56 vars in 1122 bytes} [Tue Jun 21 18:13:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:55.932823 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede0a8d-14b9-478e-9bcd-770321f0a5af None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:56.143590 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:56.158150 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede0a8d-14b9-478e-9bcd-770321f0a5af None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:56.228843 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7af964-dcf8-4c01-b86a-7f698c7ab08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:56.230112 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7af964-dcf8-4c01-b86a-7f698c7ab08e None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:56.230601 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7af964-dcf8-4c01-b86a-7f698c7ab08e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:56.231091 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7af964-dcf8-4c01-b86a-7f698c7ab08e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:56.232558 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7af964-dcf8-4c01-b86a-7f698c7ab08e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:56.235152 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7af964-dcf8-4c01-b86a-7f698c7ab08e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:56.272080 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 472/960] 10.222.0.87 () {60 vars in 1144 bytes} [Tue Jun 21 18:13:55 2022] POST /identity/v3/endpoints => generated 349 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:13:56.277417 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ae967-df7f-420c-8605-46449c30b95e None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:56.277946 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ae967-df7f-420c-8605-46449c30b95e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:56.278246 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ae967-df7f-420c-8605-46449c30b95e None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:56.287221 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 489/961] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:13:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:13:56.290046 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ade3f-fbd4-4c7b-903d-3740cb6bf4ec None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:56.290533 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ade3f-fbd4-4c7b-903d-3740cb6bf4ec None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:56.291004 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ade3f-fbd4-4c7b-903d-3740cb6bf4ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:56.325985 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-4e0ade3f-fbd4-4c7b-903d-3740cb6bf4ec None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:13:56.328202 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4e0ade3f-fbd4-4c7b-903d-3740cb6bf4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:13:56.491060 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 473/962] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:13:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:13:56.533368 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ae967-df7f-420c-8605-46449c30b95e None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:56.706222 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:56.729389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ae967-df7f-420c-8605-46449c30b95e None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:56.787964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bc2a17-b24f-4235-b5b7-0a46f66c915c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:56.788881 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bc2a17-b24f-4235-b5b7-0a46f66c915c None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:56.789272 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bc2a17-b24f-4235-b5b7-0a46f66c915c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:56.789568 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bc2a17-b24f-4235-b5b7-0a46f66c915c None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:56.790598 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bc2a17-b24f-4235-b5b7-0a46f66c915c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:56.792789 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bc2a17-b24f-4235-b5b7-0a46f66c915c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:56.813142 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.sql.core [None req-33bc2a17-b24f-4235-b5b7-0a46f66c915c None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 21 18:13:56.813142 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 21 18:13:56.813142 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [parameters: {'user_id': '70209e101ef24af88b10fcba9e77a439', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 21 18:13:56.813142 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80400) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 21 18:13:56.828858 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-33bc2a17-b24f-4235-b5b7-0a46f66c915c None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 21 18:13:56.828858 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:56.828858 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 21 18:13:56.828858 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 21 18:13:56.828858 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 21 18:13:56.828858 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 21 18:13:56.828858 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 21 18:13:56.828858 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application result = self._query(query) Jun 21 18:13:56.828858 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 21 18:13:56.828858 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application conn.query(q) Jun 21 18:13:56.828858 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 21 18:13:56.828858 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 21 18:13:56.828858 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 21 18:13:56.828858 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application result.read() Jun 21 18:13:56.828858 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 21 18:13:56.830688 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 21 18:13:56.830688 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 21 18:13:56.830688 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 21 18:13:56.830688 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 21 18:13:56.830688 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 21 18:13:56.830688 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 21 18:13:56.830688 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 21 18:13:56.830688 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 21 18:13:56.830688 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:56.830688 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 21 18:13:56.830688 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:56.830688 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:56.830688 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 21 18:13:56.830688 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 21 18:13:56.830688 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 21 18:13:56.830688 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 21 18:13:56.830688 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 21 18:13:56.830688 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application next(self.gen) Jun 21 18:13:56.832647 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 21 18:13:56.832647 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application yield resource Jun 21 18:13:56.832647 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 21 18:13:56.832647 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application next(self.gen) Jun 21 18:13:56.832647 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 21 18:13:56.832647 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self.session.rollback() Jun 21 18:13:56.832647 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 21 18:13:56.832647 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self.force_reraise() Jun 21 18:13:56.832647 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 21 18:13:56.832647 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise self.value Jun 21 18:13:56.832647 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 21 18:13:56.832647 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 21 18:13:56.832647 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 21 18:13:56.832647 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application session.commit() Jun 21 18:13:56.832647 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 21 18:13:56.832647 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 21 18:13:56.832647 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 21 18:13:56.836058 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._prepare_impl() Jun 21 18:13:56.836058 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 21 18:13:56.836058 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self.session.flush() Jun 21 18:13:56.836058 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 21 18:13:56.836058 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._flush(objects) Jun 21 18:13:56.836058 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 21 18:13:56.836058 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 21 18:13:56.836058 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 21 18:13:56.836058 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application compat.raise_( Jun 21 18:13:56.836058 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 21 18:13:56.836058 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception Jun 21 18:13:56.836058 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 21 18:13:56.836058 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application flush_context.execute() Jun 21 18:13:56.836058 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 21 18:13:56.836058 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rec.execute(self) Jun 21 18:13:56.836058 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 21 18:13:56.836058 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 21 18:13:56.839520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 21 18:13:56.839520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 21 18:13:56.839520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 21 18:13:56.839520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 21 18:13:56.839520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 21 18:13:56.839520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 21 18:13:56.839520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 21 18:13:56.839520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 21 18:13:56.839520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 21 18:13:56.839520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 21 18:13:56.839520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 21 18:13:56.839520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 21 18:13:56.839520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 21 18:13:56.839520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 21 18:13:56.839520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 21 18:13:56.839520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception Jun 21 18:13:56.847143 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 21 18:13:56.847143 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 21 18:13:56.847143 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 21 18:13:56.847143 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 21 18:13:56.847143 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 21 18:13:56.847143 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application result = self._query(query) Jun 21 18:13:56.847143 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 21 18:13:56.847143 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application conn.query(q) Jun 21 18:13:56.847143 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 21 18:13:56.847143 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 21 18:13:56.847143 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 21 18:13:56.847143 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application result.read() Jun 21 18:13:56.847143 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 21 18:13:56.847143 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 21 18:13:56.847143 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 21 18:13:56.847143 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 21 18:13:56.848847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 21 18:13:56.848847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 21 18:13:56.848847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 21 18:13:56.848847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 21 18:13:56.848847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 21 18:13:56.848847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 21 18:13:56.848847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [parameters: {'user_id': '70209e101ef24af88b10fcba9e77a439', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 21 18:13:56.848847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 21 18:13:56.848847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:56.848847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:13:56.848847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:56.848847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:56.848847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:56.848847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:56.848847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:56.850548 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:56.850548 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:56.850548 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:56.850548 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:56.850548 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:56.850548 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:56.850548 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:56.850548 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 21 18:13:56.850548 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 21 18:13:56.850548 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:56.850548 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:56.850548 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:13:56.850548 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:13:56.850548 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:13:56.850548 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:13:56.850548 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 21 18:13:56.850548 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 21 18:13:56.852718 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 21 18:13:56.852718 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 21 18:13:56.852718 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 21 18:13:56.852718 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 21 18:13:56.852718 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 21 18:13:56.852718 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:56.852718 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 490/963] 10.222.0.87 () {60 vars in 1131 bytes} [Tue Jun 21 18:13:56 2022] POST /identity/v3/users => generated 160 bytes in 299 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:56.853476 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0ade3f-fbd4-4c7b-903d-3740cb6bf4ec None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:57.034334 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:57.087071 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0ade3f-fbd4-4c7b-903d-3740cb6bf4ec None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:57.158558 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf42da-2e07-4b86-ac29-95a6298b266b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:57.159903 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf42da-2e07-4b86-ac29-95a6298b266b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:57.160418 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf42da-2e07-4b86-ac29-95a6298b266b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:57.161326 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf42da-2e07-4b86-ac29-95a6298b266b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:57.175338 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cf42da-2e07-4b86-ac29-95a6298b266b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:57.178171 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cf42da-2e07-4b86-ac29-95a6298b266b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:57.208284 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-47cf42da-2e07-4b86-ac29-95a6298b266b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 21 18:13:57.208284 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:57.208284 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:57.208284 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:57.208284 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:57.208284 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:57.208284 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:57.208284 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:57.208284 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:57.208284 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:57.208284 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:57.208284 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:57.208284 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:13:57.208284 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:13:57.208284 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:13:57.208284 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:13:57.209299 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:57.209299 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:57.209299 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:13:57.209299 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:13:57.209299 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:13:57.209299 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:13:57.209299 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:57.209299 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:57.209299 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:57.209299 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:57.209299 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:57.209299 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:57.209299 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:57.209299 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:57.209299 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:57.209299 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:57.209299 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:57.209299 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:57.210285 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:57.210285 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:57.210285 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:13:57.210285 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:13:57.210285 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:13:57.210285 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:13:57.210285 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:13:57.210285 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:13:57.210285 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 21 18:13:57.210285 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:57.211473 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 474/964] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:13:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:57.218533 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bc2a17-b24f-4235-b5b7-0a46f66c915c None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:57.402409 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bc2a17-b24f-4235-b5b7-0a46f66c915c None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:57.454446 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8707b506-3434-4221-b665-bdf99ea28751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:57.455655 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707b506-3434-4221-b665-bdf99ea28751 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:57.456411 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707b506-3434-4221-b665-bdf99ea28751 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:57.457936 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707b506-3434-4221-b665-bdf99ea28751 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:57.460021 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8707b506-3434-4221-b665-bdf99ea28751 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:57.460880 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8707b506-3434-4221-b665-bdf99ea28751 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:57.464344 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8707b506-3434-4221-b665-bdf99ea28751 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:57.549820 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 491/965] 10.222.0.87 () {56 vars in 1101 bytes} [Tue Jun 21 18:13:57 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:58.433480 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2181791-03bc-4000-ae01-f74c1992a301 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:58.434088 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2181791-03bc-4000-ae01-f74c1992a301 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:58.434485 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2181791-03bc-4000-ae01-f74c1992a301 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:58.437312 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 475/966] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:13:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:13:58.446713 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6351c2-0f86-45a6-82c3-c5526cfe7533 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:58.447211 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6351c2-0f86-45a6-82c3-c5526cfe7533 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:58.447719 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6351c2-0f86-45a6-82c3-c5526cfe7533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:58.475488 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-ca6351c2-0f86-45a6-82c3-c5526cfe7533 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:13:58.477401 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ca6351c2-0f86-45a6-82c3-c5526cfe7533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:13:58.480715 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:58.650094 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 492/967] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:13:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:13:58.694380 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2181791-03bc-4000-ae01-f74c1992a301 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:58.982825 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2181791-03bc-4000-ae01-f74c1992a301 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:59.054756 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac3db7a-93ad-43ee-97a4-0707a5911c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:59.055875 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3db7a-93ad-43ee-97a4-0707a5911c8e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:59.057796 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3db7a-93ad-43ee-97a4-0707a5911c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:59.057796 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3db7a-93ad-43ee-97a4-0707a5911c8e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:59.065000 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:59.086376 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac3db7a-93ad-43ee-97a4-0707a5911c8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:13:59.098859 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-dac3db7a-93ad-43ee-97a4-0707a5911c8e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 18:13:59.098859 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:59.098859 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:59.098859 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:59.098859 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:59.098859 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:59.098859 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:59.098859 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:59.098859 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:59.098859 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:59.098859 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:59.098859 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:59.098859 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:13:59.098859 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:13:59.098859 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:13:59.098859 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:13:59.100244 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:13:59.100244 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:13:59.100244 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:13:59.100244 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:13:59.100244 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:59.100244 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:59.100244 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:59.100244 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:59.100244 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:59.100244 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:59.100244 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:59.100244 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:59.100244 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:59.100244 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:59.100244 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:59.100244 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:59.101709 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:59.101709 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:59.101709 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:59.101709 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:59.101709 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:13:59.101709 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:13:59.101709 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:13:59.101709 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:13:59.101709 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:13:59.101709 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:13:59.101709 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 18:13:59.101709 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:13:59.101709 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 476/968] 10.222.0.87 () {56 vars in 1120 bytes} [Tue Jun 21 18:13:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 408 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:59.110659 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6351c2-0f86-45a6-82c3-c5526cfe7533 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:59.311020 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86554be0-cb00-43dc-a30f-5de57427e01b None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:59.311492 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86554be0-cb00-43dc-a30f-5de57427e01b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:59.311904 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86554be0-cb00-43dc-a30f-5de57427e01b None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:59.315080 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 477/969] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:13:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:13:59.323896 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c29aac-d07b-4713-bb5f-3e6e49150ef1 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:59.324376 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c29aac-d07b-4713-bb5f-3e6e49150ef1 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:59.324882 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c29aac-d07b-4713-bb5f-3e6e49150ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:59.358024 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-c1c29aac-d07b-4713-bb5f-3e6e49150ef1 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:13:59.363761 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6351c2-0f86-45a6-82c3-c5526cfe7533 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:59.364587 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c1c29aac-d07b-4713-bb5f-3e6e49150ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:13:59.443086 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d2da4d-e981-4879-86e7-0baa7dbb3b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:59.444159 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d2da4d-e981-4879-86e7-0baa7dbb3b5d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:59.444679 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d2da4d-e981-4879-86e7-0baa7dbb3b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:59.445146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d2da4d-e981-4879-86e7-0baa7dbb3b5d None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:59.446565 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d2da4d-e981-4879-86e7-0baa7dbb3b5d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:13:59.447279 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d2da4d-e981-4879-86e7-0baa7dbb3b5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:59.450103 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d2da4d-e981-4879-86e7-0baa7dbb3b5d None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:59.464457 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:59.474269 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 493/970] 10.222.0.87 () {56 vars in 1115 bytes} [Tue Jun 21 18:13:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:13:59.489105 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d2da4d-e981-4879-86e7-0baa7dbb3b5d None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:59.539006 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 478/971] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:13:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:13:59.581389 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c29aac-d07b-4713-bb5f-3e6e49150ef1 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:59.734608 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d2da4d-e981-4879-86e7-0baa7dbb3b5d None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:59.827904 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fa8cf9-4608-4b63-8609-3635f1f7224f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:59.829377 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa8cf9-4608-4b63-8609-3635f1f7224f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:59.830098 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa8cf9-4608-4b63-8609-3635f1f7224f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:59.830435 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa8cf9-4608-4b63-8609-3635f1f7224f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:59.860111 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c29aac-d07b-4713-bb5f-3e6e49150ef1 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:13:59.861490 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fa8cf9-4608-4b63-8609-3635f1f7224f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:59.865970 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fa8cf9-4608-4b63-8609-3635f1f7224f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:59.906731 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:13:59.908472 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-c6fa8cf9-4608-4b63-8609-3635f1f7224f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:59.908472 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:13:59.908472 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:13:59.908472 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:13:59.908472 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:13:59.908472 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:13:59.908472 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:13:59.908472 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:13:59.908472 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:13:59.908472 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:13:59.908472 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:13:59.908472 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:13:59.908472 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:13:59.908472 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:13:59.908472 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:13:59.911133 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:13:59.911133 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:13:59.911133 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:13:59.911133 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:13:59.911133 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:13:59.911133 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:13:59.911133 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:13:59.911133 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:13:59.911133 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:13:59.911133 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:13:59.911133 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:13:59.911133 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:13:59.911133 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:13:59.911133 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:13:59.911133 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:13:59.911133 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:13:59.911133 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:13:59.913076 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:13:59.913076 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:13:59.913076 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:13:59.913076 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:13:59.917912 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 494/972] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:13:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 434 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:13:59.921497 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fa8cf9-4608-4b63-8609-3635f1f7224f None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:13:59.940839 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c86b19c-b05e-4947-b206-3ec0e782683c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:13:59.942279 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c86b19c-b05e-4947-b206-3ec0e782683c None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:59.942821 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c86b19c-b05e-4947-b206-3ec0e782683c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:59.943252 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c86b19c-b05e-4947-b206-3ec0e782683c None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:59.944566 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c86b19c-b05e-4947-b206-3ec0e782683c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:13:59.947269 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c86b19c-b05e-4947-b206-3ec0e782683c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:13:59.974707 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 479/973] 10.222.0.87 () {60 vars in 1141 bytes} [Tue Jun 21 18:13:59 2022] POST /identity/v3/services => generated 242 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:13:59.977150 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a7c5b-4216-41f6-922c-19df3bda0503 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:59.977651 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a7c5b-4216-41f6-922c-19df3bda0503 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:59.978205 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a7c5b-4216-41f6-922c-19df3bda0503 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:13:59.985716 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 480/974] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:13:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:13:59.988872 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac80d0-87b3-4f36-a0e2-eac94d5ab042 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:13:59.989483 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac80d0-87b3-4f36-a0e2-eac94d5ab042 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:13:59.989950 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac80d0-87b3-4f36-a0e2-eac94d5ab042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:00.017494 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-4eac80d0-87b3-4f36-a0e2-eac94d5ab042 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:14:00.019710 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4eac80d0-87b3-4f36-a0e2-eac94d5ab042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:14:00.073746 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:00.203964 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 481/975] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:13:59 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:14:00.223514 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fa8cf9-4608-4b63-8609-3635f1f7224f None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:00.242279 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac80d0-87b3-4f36-a0e2-eac94d5ab042 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:00.315114 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8540fae3-e8fd-497c-8bc9-8527b6d92f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:00.316856 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8540fae3-e8fd-497c-8bc9-8527b6d92f28 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:00.317373 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8540fae3-e8fd-497c-8bc9-8527b6d92f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:00.317901 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8540fae3-e8fd-497c-8bc9-8527b6d92f28 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:00.319303 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8540fae3-e8fd-497c-8bc9-8527b6d92f28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:00.319830 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8540fae3-e8fd-497c-8bc9-8527b6d92f28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:00.322423 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8540fae3-e8fd-497c-8bc9-8527b6d92f28 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:00.349238 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 495/976] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:13:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 429 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:14:00.358740 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8540fae3-e8fd-497c-8bc9-8527b6d92f28 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:00.485782 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac80d0-87b3-4f36-a0e2-eac94d5ab042 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:00.507874 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:00.562176 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee623b21-1495-4a3b-a479-4ac3432ffda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:00.563609 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee623b21-1495-4a3b-a479-4ac3432ffda6 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:00.564105 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee623b21-1495-4a3b-a479-4ac3432ffda6 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:00.564610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee623b21-1495-4a3b-a479-4ac3432ffda6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:00.565864 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee623b21-1495-4a3b-a479-4ac3432ffda6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:00.568513 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee623b21-1495-4a3b-a479-4ac3432ffda6 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:00.593249 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-ee623b21-1495-4a3b-a479-4ac3432ffda6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 18:14:00.593249 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:14:00.593249 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:14:00.593249 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:14:00.593249 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:14:00.593249 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:14:00.593249 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:14:00.593249 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:14:00.593249 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 18:14:00.593249 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:14:00.593249 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:14:00.593249 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:14:00.593249 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:14:00.593249 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:14:00.593249 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:14:00.593249 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:14:00.593249 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:14:00.595550 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:14:00.595550 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:14:00.595550 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:14:00.595550 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:14:00.595550 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:14:00.595550 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:14:00.595550 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:14:00.595550 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:14:00.595550 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:14:00.595550 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:14:00.595550 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:14:00.595550 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:14:00.595550 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:14:00.595550 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:14:00.595550 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:14:00.595550 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:14:00.596555 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:14:00.596555 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:14:00.596555 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:14:00.596555 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:14:00.596555 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:14:00.596555 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:14:00.596555 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:14:00.596555 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:14:00.596555 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:14:00.596555 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:14:00.596555 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:14:00.596555 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:14:00.596555 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 18:14:00.596555 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:14:00.596555 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 482/977] 10.222.0.87 () {56 vars in 1126 bytes} [Tue Jun 21 18:14:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:14:00.605480 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee623b21-1495-4a3b-a479-4ac3432ffda6 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:00.620978 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8540fae3-e8fd-497c-8bc9-8527b6d92f28 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:00.652845 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:00.705887 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eec5d6-227b-48bd-8693-441edb2facd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:00.707013 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eec5d6-227b-48bd-8693-441edb2facd9 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:00.707502 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eec5d6-227b-48bd-8693-441edb2facd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:00.708276 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eec5d6-227b-48bd-8693-441edb2facd9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:00.722641 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eec5d6-227b-48bd-8693-441edb2facd9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:00.725335 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eec5d6-227b-48bd-8693-441edb2facd9 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:00.748887 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-45eec5d6-227b-48bd-8693-441edb2facd9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 18:14:00.748887 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:14:00.748887 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:14:00.748887 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:14:00.748887 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:14:00.748887 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:14:00.748887 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:14:00.748887 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:14:00.748887 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:14:00.748887 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:14:00.748887 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:14:00.748887 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:14:00.748887 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:14:00.748887 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:14:00.748887 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:14:00.748887 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:14:00.753083 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:14:00.753083 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:14:00.753083 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:14:00.753083 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:14:00.753083 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:14:00.753083 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:14:00.753083 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:14:00.753083 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:14:00.753083 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:14:00.753083 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:14:00.753083 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:14:00.753083 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:14:00.753083 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:14:00.753083 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:14:00.753083 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:14:00.753083 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:14:00.753083 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:14:00.753083 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:14:00.755137 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:14:00.755137 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:14:00.755137 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:14:00.755137 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:14:00.755137 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:14:00.755137 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:14:00.755137 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:14:00.755137 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:14:00.755137 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 18:14:00.755137 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:14:00.755137 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 496/978] 10.222.0.87 () {56 vars in 1117 bytes} [Tue Jun 21 18:14:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 393 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:14:00.758442 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eec5d6-227b-48bd-8693-441edb2facd9 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:00.838138 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee623b21-1495-4a3b-a479-4ac3432ffda6 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:00.932194 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf40e3dd-5578-4bc5-8300-36756438ca29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:00.933312 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf40e3dd-5578-4bc5-8300-36756438ca29 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:00.933869 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf40e3dd-5578-4bc5-8300-36756438ca29 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:00.934468 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf40e3dd-5578-4bc5-8300-36756438ca29 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:00.935902 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf40e3dd-5578-4bc5-8300-36756438ca29 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:00.937341 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf40e3dd-5578-4bc5-8300-36756438ca29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:00.938896 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf40e3dd-5578-4bc5-8300-36756438ca29 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:00.957739 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 483/979] 10.222.0.87 () {56 vars in 1122 bytes} [Tue Jun 21 18:14:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:14:00.967035 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf40e3dd-5578-4bc5-8300-36756438ca29 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:00.999665 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eec5d6-227b-48bd-8693-441edb2facd9 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:01.051421 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:01.064828 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936472b2-e5fa-4f4e-bb23-291f369df6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:01.065823 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936472b2-e5fa-4f4e-bb23-291f369df6f8 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:01.066179 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936472b2-e5fa-4f4e-bb23-291f369df6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:01.066503 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936472b2-e5fa-4f4e-bb23-291f369df6f8 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:01.067802 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936472b2-e5fa-4f4e-bb23-291f369df6f8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:01.068412 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936472b2-e5fa-4f4e-bb23-291f369df6f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:01.070392 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936472b2-e5fa-4f4e-bb23-291f369df6f8 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:01.104915 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 497/980] 10.222.0.87 () {56 vars in 1149 bytes} [Tue Jun 21 18:14:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:14:01.112532 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936472b2-e5fa-4f4e-bb23-291f369df6f8 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:01.185888 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:01.205899 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf40e3dd-5578-4bc5-8300-36756438ca29 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:01.289295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2d2eaf-a024-415a-9f96-76faf17e75a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:01.290369 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d2eaf-a024-415a-9f96-76faf17e75a8 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:01.290683 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d2eaf-a024-415a-9f96-76faf17e75a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:01.290965 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d2eaf-a024-415a-9f96-76faf17e75a8 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:01.292086 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2d2eaf-a024-415a-9f96-76faf17e75a8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:01.292418 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2d2eaf-a024-415a-9f96-76faf17e75a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:01.294339 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2d2eaf-a024-415a-9f96-76faf17e75a8 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:01.310578 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 484/981] 10.222.0.87 () {56 vars in 1122 bytes} [Tue Jun 21 18:14:00 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:14:01.311820 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2d2eaf-a024-415a-9f96-76faf17e75a8 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:01.358499 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936472b2-e5fa-4f4e-bb23-291f369df6f8 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:01.440591 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a92a519-c5e1-42f0-a2d7-c3cb1be6930c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:01.441721 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92a519-c5e1-42f0-a2d7-c3cb1be6930c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:01.442192 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92a519-c5e1-42f0-a2d7-c3cb1be6930c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:01.442679 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92a519-c5e1-42f0-a2d7-c3cb1be6930c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:01.544919 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2d2eaf-a024-415a-9f96-76faf17e75a8 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:01.549497 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:01.592794 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a92a519-c5e1-42f0-a2d7-c3cb1be6930c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:01.595215 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a92a519-c5e1-42f0-a2d7-c3cb1be6930c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:01.638075 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddfd1ad-5ded-4781-86d0-8b3922962ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:01.639361 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddfd1ad-5ded-4781-86d0-8b3922962ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:01.639700 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddfd1ad-5ded-4781-86d0-8b3922962ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:01.640235 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddfd1ad-5ded-4781-86d0-8b3922962ba2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:01.641814 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddfd1ad-5ded-4781-86d0-8b3922962ba2 None admin] RBAC: Adding query filter params (interface=admin, service_id=cb8fc51a0dfa4523b6309034c0c5cadd, region_id=RegionOne) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:01.642377 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddfd1ad-5ded-4781-86d0-8b3922962ba2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:01.645065 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddfd1ad-5ded-4781-86d0-8b3922962ba2 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:01.655905 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 485/982] 10.222.0.87 () {56 vars in 1249 bytes} [Tue Jun 21 18:14:01 2022] GET /identity/v3/endpoints?service_id=cb8fc51a0dfa4523b6309034c0c5cadd&interface=admin®ion_id=RegionOne => generated 193 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:14:01.664748 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:01.672357 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddfd1ad-5ded-4781-86d0-8b3922962ba2 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:01.851615 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 498/983] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:14:01 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 740 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:14:01.860997 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a92a519-c5e1-42f0-a2d7-c3cb1be6930c None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:01.924516 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddfd1ad-5ded-4781-86d0-8b3922962ba2 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:02.004824 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7220dbcd-a581-4995-ac16-884142c7b7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:02.005987 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220dbcd-a581-4995-ac16-884142c7b7a1 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:02.006459 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220dbcd-a581-4995-ac16-884142c7b7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:02.006915 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220dbcd-a581-4995-ac16-884142c7b7a1 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:02.008052 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7220dbcd-a581-4995-ac16-884142c7b7a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:02.010701 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7220dbcd-a581-4995-ac16-884142c7b7a1 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:02.031574 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 486/984] 10.222.0.87 () {56 vars in 1087 bytes} [Tue Jun 21 18:14:01 2022] GET /identity/v3/services => generated 1727 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:14:02.103715 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a92a519-c5e1-42f0-a2d7-c3cb1be6930c None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:02.185479 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:02.201069 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ade729-6f4c-435a-89a6-19dda19c2e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:02.202466 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ade729-6f4c-435a-89a6-19dda19c2e5a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:02.202953 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ade729-6f4c-435a-89a6-19dda19c2e5a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:02.203440 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ade729-6f4c-435a-89a6-19dda19c2e5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:02.225092 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ade729-6f4c-435a-89a6-19dda19c2e5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:02.227887 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ade729-6f4c-435a-89a6-19dda19c2e5a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:02.264944 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-43ade729-6f4c-435a-89a6-19dda19c2e5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:14:02.264944 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:14:02.264944 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:14:02.264944 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:14:02.264944 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:14:02.264944 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:14:02.264944 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:14:02.264944 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:14:02.264944 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:14:02.264944 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:14:02.264944 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:14:02.264944 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:14:02.264944 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:14:02.264944 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:14:02.264944 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:14:02.267322 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:14:02.267322 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:14:02.267322 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:14:02.267322 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:14:02.267322 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:14:02.267322 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:14:02.267322 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:14:02.267322 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:14:02.267322 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:14:02.267322 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:14:02.267322 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:14:02.267322 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:14:02.267322 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:14:02.267322 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:14:02.267322 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:14:02.267322 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:14:02.267322 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:14:02.269111 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:14:02.269111 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:14:02.269111 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:14:02.269111 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:14:02.274199 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 499/985] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:14:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 414 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:14:02.278035 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7220dbcd-a581-4995-ac16-884142c7b7a1 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:02.370005 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:02.554614 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7220dbcd-a581-4995-ac16-884142c7b7a1 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:02.653952 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b992cba-743a-4f1d-af79-eaabfce81739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:02.655106 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b992cba-743a-4f1d-af79-eaabfce81739 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:02.655546 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b992cba-743a-4f1d-af79-eaabfce81739 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:02.656038 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b992cba-743a-4f1d-af79-eaabfce81739 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:02.657993 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b992cba-743a-4f1d-af79-eaabfce81739 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:02.658200 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b992cba-743a-4f1d-af79-eaabfce81739 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:02.662052 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b992cba-743a-4f1d-af79-eaabfce81739 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:02.703406 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 487/986] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:14:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 427 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:14:02.714245 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ade729-6f4c-435a-89a6-19dda19c2e5a None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:02.999894 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ade729-6f4c-435a-89a6-19dda19c2e5a None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:03.085103 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc037376-c974-4121-a9d7-b9a8cc870118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:03.086529 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc037376-c974-4121-a9d7-b9a8cc870118 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:03.087038 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc037376-c974-4121-a9d7-b9a8cc870118 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:03.087525 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc037376-c974-4121-a9d7-b9a8cc870118 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:03.110228 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc037376-c974-4121-a9d7-b9a8cc870118 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:03.113138 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc037376-c974-4121-a9d7-b9a8cc870118 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:03.151255 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-cc037376-c974-4121-a9d7-b9a8cc870118 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:14:03.151255 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:14:03.151255 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:14:03.151255 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:14:03.151255 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:14:03.151255 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:14:03.151255 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:14:03.151255 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:14:03.151255 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:14:03.151255 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:14:03.151255 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:14:03.151255 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:14:03.151255 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:14:03.151255 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:14:03.151255 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:14:03.153047 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:14:03.153047 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:14:03.153047 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:14:03.153047 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:14:03.153047 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:14:03.153047 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:14:03.153047 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:14:03.153047 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:14:03.153047 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:14:03.153047 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:14:03.153047 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:14:03.153047 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:14:03.153047 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:14:03.153047 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:14:03.153047 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:14:03.153047 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:14:03.153047 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:14:03.155418 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:14:03.155418 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:14:03.155418 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:14:03.155418 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:14:03.155418 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:14:03.155418 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:14:03.155418 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:14:03.155418 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:14:03.155418 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 500/987] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:14:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 441 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:14:03.163818 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b992cba-743a-4f1d-af79-eaabfce81739 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:03.412756 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:03.433521 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b992cba-743a-4f1d-af79-eaabfce81739 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:03.447874 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11fdb49-a6f7-410d-a95b-5b316c884ac2 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:03.448390 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11fdb49-a6f7-410d-a95b-5b316c884ac2 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:03.448909 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11fdb49-a6f7-410d-a95b-5b316c884ac2 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:03.451900 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 501/988] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:14:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:14:03.463142 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1a735-d4ca-4edc-ac15-8588a7c4b698 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:03.464611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1a735-d4ca-4edc-ac15-8588a7c4b698 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:03.472120 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1a735-d4ca-4edc-ac15-8588a7c4b698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:03.504238 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-1af1a735-d4ca-4edc-ac15-8588a7c4b698 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:14:03.512745 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4a5005-608e-4859-985e-f5c359862267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:03.513960 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a5005-608e-4859-985e-f5c359862267 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:03.515251 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-1af1a735-d4ca-4edc-ac15-8588a7c4b698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:14:03.515422 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a5005-608e-4859-985e-f5c359862267 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:03.515422 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a5005-608e-4859-985e-f5c359862267 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:03.517199 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4a5005-608e-4859-985e-f5c359862267 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:03.517199 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4a5005-608e-4859-985e-f5c359862267 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:03.520579 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4a5005-608e-4859-985e-f5c359862267 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:03.549045 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 488/989] 10.222.0.87 () {56 vars in 1148 bytes} [Tue Jun 21 18:14:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:14:03.559355 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4a5005-608e-4859-985e-f5c359862267 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:03.614848 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:03.702516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 502/990] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:14:03 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:14:03.732959 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af1a735-d4ca-4edc-ac15-8588a7c4b698 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:03.823989 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4a5005-608e-4859-985e-f5c359862267 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:03.918301 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a115cfa-e134-4895-bc72-3b9b268a1764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:03.919825 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a115cfa-e134-4895-bc72-3b9b268a1764 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:03.920624 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a115cfa-e134-4895-bc72-3b9b268a1764 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:03.921338 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a115cfa-e134-4895-bc72-3b9b268a1764 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:03.923396 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a115cfa-e134-4895-bc72-3b9b268a1764 None admin] RBAC: Adding query filter params (role.id=9fa867d83f3c406aabf51ddd8c7ed2aa, scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb, user.id=09c9fa7dccb8491c9ad3b43967345ea2) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:03.924199 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a115cfa-e134-4895-bc72-3b9b268a1764 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:03.928264 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a115cfa-e134-4895-bc72-3b9b268a1764 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:03.951285 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 489/991] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:14:03 2022] GET /identity/v3/role_assignments?user.id=09c9fa7dccb8491c9ad3b43967345ea2&scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb&role.id=9fa867d83f3c406aabf51ddd8c7ed2aa => generated 608 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:14:04.018625 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af1a735-d4ca-4edc-ac15-8588a7c4b698 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:04.113738 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c5b84e-408a-4750-9134-7f074b1f7f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:04.115177 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c5b84e-408a-4750-9134-7f074b1f7f85 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:04.115691 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c5b84e-408a-4750-9134-7f074b1f7f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:04.116286 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c5b84e-408a-4750-9134-7f074b1f7f85 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:04.117677 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c5b84e-408a-4750-9134-7f074b1f7f85 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:04.120524 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c5b84e-408a-4750-9134-7f074b1f7f85 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:04.147868 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-22c5b84e-408a-4750-9134-7f074b1f7f85 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 21 18:14:04.147868 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:14:04.147868 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:14:04.147868 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:14:04.147868 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:14:04.147868 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:14:04.147868 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:14:04.147868 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:14:04.147868 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 21 18:14:04.147868 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:14:04.147868 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:14:04.147868 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:14:04.147868 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:14:04.147868 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:14:04.147868 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:14:04.147868 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:14:04.147868 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:14:04.149783 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:14:04.149783 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:14:04.149783 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:14:04.149783 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:14:04.149783 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:14:04.149783 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:14:04.149783 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:14:04.149783 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:14:04.149783 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:14:04.149783 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:14:04.149783 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:14:04.149783 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:14:04.149783 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:14:04.149783 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:14:04.149783 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:14:04.149783 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:14:04.159604 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:14:04.159604 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:14:04.159604 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:14:04.159604 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:14:04.159604 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:14:04.159604 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:14:04.159604 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:14:04.159604 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:14:04.159604 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:14:04.159604 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:14:04.159604 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:14:04.159604 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:14:04.159604 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 21 18:14:04.159604 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:14:04.159604 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 503/992] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:14:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 420 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:14:04.164892 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a115cfa-e134-4895-bc72-3b9b268a1764 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:04.213913 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:04.433025 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a115cfa-e134-4895-bc72-3b9b268a1764 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:04.513493 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15eb1f5-10cb-4328-aa9f-81e05b7650fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:04.514794 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15eb1f5-10cb-4328-aa9f-81e05b7650fa None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:04.515300 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15eb1f5-10cb-4328-aa9f-81e05b7650fa None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:04.515789 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15eb1f5-10cb-4328-aa9f-81e05b7650fa None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:04.517055 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15eb1f5-10cb-4328-aa9f-81e05b7650fa None admin] RBAC: Adding query filter params (name=compute) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:04.517539 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15eb1f5-10cb-4328-aa9f-81e05b7650fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:04.520488 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15eb1f5-10cb-4328-aa9f-81e05b7650fa None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:04.535639 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 490/993] 10.222.0.87 () {56 vars in 1112 bytes} [Tue Jun 21 18:14:04 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:14:04.541059 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c5b84e-408a-4750-9134-7f074b1f7f85 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:04.634736 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:04.802896 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c5b84e-408a-4750-9134-7f074b1f7f85 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:04.896969 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be80a282-cb59-4b3f-b281-2baf5e61ded2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:04.898107 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be80a282-cb59-4b3f-b281-2baf5e61ded2 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:04.898598 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be80a282-cb59-4b3f-b281-2baf5e61ded2 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:04.899094 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be80a282-cb59-4b3f-b281-2baf5e61ded2 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:04.900573 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be80a282-cb59-4b3f-b281-2baf5e61ded2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:04.901181 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be80a282-cb59-4b3f-b281-2baf5e61ded2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:04.904056 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be80a282-cb59-4b3f-b281-2baf5e61ded2 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:04.929899 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 504/994] 10.222.0.87 () {56 vars in 1112 bytes} [Tue Jun 21 18:14:04 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:14:04.933036 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15eb1f5-10cb-4328-aa9f-81e05b7650fa None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:05.132317 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:05.180992 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15eb1f5-10cb-4328-aa9f-81e05b7650fa None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:05.250793 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873f0f4b-609d-40d8-b5a3-d39da265f113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:05.251674 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f0f4b-609d-40d8-b5a3-d39da265f113 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:05.251968 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f0f4b-609d-40d8-b5a3-d39da265f113 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:05.252323 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f0f4b-609d-40d8-b5a3-d39da265f113 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:05.253299 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873f0f4b-609d-40d8-b5a3-d39da265f113 None admin] RBAC: Adding query filter params (interface=public, service_id=9712e293b90d4757a6af68b22d6ed758, region_id=RegionOne) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:05.253625 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873f0f4b-609d-40d8-b5a3-d39da265f113 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:05.255381 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873f0f4b-609d-40d8-b5a3-d39da265f113 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:05.262668 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 491/995] 10.222.0.87 () {56 vars in 1251 bytes} [Tue Jun 21 18:14:04 2022] GET /identity/v3/endpoints?service_id=9712e293b90d4757a6af68b22d6ed758&interface=public®ion_id=RegionOne => generated 194 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:14:05.270739 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be80a282-cb59-4b3f-b281-2baf5e61ded2 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:05.314776 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b837346-4812-4edc-b66d-cf69b1dd6191 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:05.316163 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b837346-4812-4edc-b66d-cf69b1dd6191 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:05.316163 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b837346-4812-4edc-b66d-cf69b1dd6191 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:05.318702 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 492/996] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:14:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:14:05.325822 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a73119-20bd-4537-90cf-becdcee2c48b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:05.326280 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a73119-20bd-4537-90cf-becdcee2c48b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:05.326863 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a73119-20bd-4537-90cf-becdcee2c48b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:05.352099 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-53a73119-20bd-4537-90cf-becdcee2c48b None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:14:05.354357 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-53a73119-20bd-4537-90cf-becdcee2c48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:14:05.501875 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 493/997] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:14:05 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:14:05.513674 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:05.519375 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be80a282-cb59-4b3f-b281-2baf5e61ded2 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:05.547001 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a73119-20bd-4537-90cf-becdcee2c48b None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:05.606099 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a355b7-5a2e-480d-a851-9ae92f606780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:05.608004 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a355b7-5a2e-480d-a851-9ae92f606780 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:05.608486 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a355b7-5a2e-480d-a851-9ae92f606780 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:05.608935 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a355b7-5a2e-480d-a851-9ae92f606780 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:05.610249 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a355b7-5a2e-480d-a851-9ae92f606780 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:05.613182 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a355b7-5a2e-480d-a851-9ae92f606780 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:05.632355 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 505/998] 10.222.0.87 () {56 vars in 1087 bytes} [Tue Jun 21 18:14:05 2022] GET /identity/v3/services => generated 1727 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:14:05.730736 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:05.774495 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a73119-20bd-4537-90cf-becdcee2c48b None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:05.853750 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d6c342-cc1c-4a86-b504-97691360eeb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:05.855424 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6c342-cc1c-4a86-b504-97691360eeb5 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:05.855803 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6c342-cc1c-4a86-b504-97691360eeb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:05.856288 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6c342-cc1c-4a86-b504-97691360eeb5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:05.857711 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d6c342-cc1c-4a86-b504-97691360eeb5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:05.860509 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d6c342-cc1c-4a86-b504-97691360eeb5 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:05.885870 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-c4d6c342-cc1c-4a86-b504-97691360eeb5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 18:14:05.885870 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:14:05.885870 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:14:05.885870 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:14:05.885870 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:14:05.885870 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:14:05.885870 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:14:05.885870 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:14:05.885870 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 18:14:05.885870 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:14:05.885870 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:14:05.885870 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:14:05.885870 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:14:05.885870 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:14:05.885870 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:14:05.885870 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:14:05.885870 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:14:05.887783 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:14:05.887783 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:14:05.887783 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:14:05.887783 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:14:05.887783 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:14:05.887783 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:14:05.887783 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:14:05.887783 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:14:05.887783 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:14:05.887783 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:14:05.887783 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:14:05.887783 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:14:05.887783 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:14:05.887783 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:14:05.887783 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:14:05.887783 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:14:05.892995 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:14:05.892995 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:14:05.892995 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:14:05.892995 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:14:05.892995 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:14:05.892995 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:14:05.892995 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:14:05.892995 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:14:05.892995 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:14:05.892995 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:14:05.892995 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:14:05.892995 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:14:05.892995 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 18:14:05.892995 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:14:05.892995 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 494/999] 10.222.0.87 () {56 vars in 1126 bytes} [Tue Jun 21 18:14:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:14:05.897535 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a355b7-5a2e-480d-a851-9ae92f606780 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:06.074993 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a355b7-5a2e-480d-a851-9ae92f606780 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:06.140961 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313c3798-33d2-4db9-bfee-d7ae31373223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:06.142088 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c3798-33d2-4db9-bfee-d7ae31373223 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:06.142503 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c3798-33d2-4db9-bfee-d7ae31373223 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:06.142908 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c3798-33d2-4db9-bfee-d7ae31373223 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:06.144802 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313c3798-33d2-4db9-bfee-d7ae31373223 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:06.145266 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313c3798-33d2-4db9-bfee-d7ae31373223 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:06.147652 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313c3798-33d2-4db9-bfee-d7ae31373223 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:06.156431 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 506/1000] 10.222.0.87 () {56 vars in 1122 bytes} [Tue Jun 21 18:14:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:14:06.165784 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d6c342-cc1c-4a86-b504-97691360eeb5 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:06.405428 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d6c342-cc1c-4a86-b504-97691360eeb5 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:06.490195 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0814815e-851c-4f7e-aa47-6093e92d6494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:06.491216 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0814815e-851c-4f7e-aa47-6093e92d6494 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:06.491648 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0814815e-851c-4f7e-aa47-6093e92d6494 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:06.492069 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0814815e-851c-4f7e-aa47-6093e92d6494 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:06.493489 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0814815e-851c-4f7e-aa47-6093e92d6494 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:06.493992 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0814815e-851c-4f7e-aa47-6093e92d6494 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:06.496530 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0814815e-851c-4f7e-aa47-6093e92d6494 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:06.508112 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 495/1001] 10.222.0.87 () {56 vars in 1122 bytes} [Tue Jun 21 18:14:06 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:14:06.517174 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313c3798-33d2-4db9-bfee-d7ae31373223 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:06.588681 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:06.771602 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313c3798-33d2-4db9-bfee-d7ae31373223 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:06.857289 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c1348d-5f71-4bbe-9d5a-1442f1d40bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:06.858220 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c1348d-5f71-4bbe-9d5a-1442f1d40bcd None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:06.858635 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c1348d-5f71-4bbe-9d5a-1442f1d40bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:06.859054 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c1348d-5f71-4bbe-9d5a-1442f1d40bcd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:06.859982 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c1348d-5f71-4bbe-9d5a-1442f1d40bcd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:06.861534 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c1348d-5f71-4bbe-9d5a-1442f1d40bcd None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:06.901693 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 507/1002] 10.222.0.87 () {60 vars in 1144 bytes} [Tue Jun 21 18:14:06 2022] POST /identity/v3/endpoints => generated 325 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:14:07.543208 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad80ed9-0cde-482e-9fdc-6f915ad1b845 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:07.543784 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad80ed9-0cde-482e-9fdc-6f915ad1b845 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:07.544228 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad80ed9-0cde-482e-9fdc-6f915ad1b845 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:07.547216 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 496/1003] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:14:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:14:07.556924 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263839c8-1b69-4c55-9cf7-121d59b3c9c3 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:07.558025 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263839c8-1b69-4c55-9cf7-121d59b3c9c3 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:07.558025 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263839c8-1b69-4c55-9cf7-121d59b3c9c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:07.592172 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-263839c8-1b69-4c55-9cf7-121d59b3c9c3 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:14:07.594579 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-263839c8-1b69-4c55-9cf7-121d59b3c9c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:14:07.762785 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 508/1004] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:14:07 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:14:07.797122 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad80ed9-0cde-482e-9fdc-6f915ad1b845 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:07.803250 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:08.031249 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad80ed9-0cde-482e-9fdc-6f915ad1b845 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:08.109552 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6931cd-4ee3-4db5-ba78-4173a4278c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:08.110806 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6931cd-4ee3-4db5-ba78-4173a4278c83 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:08.111746 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6931cd-4ee3-4db5-ba78-4173a4278c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:08.111904 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6931cd-4ee3-4db5-ba78-4173a4278c83 None admin] PATH_INFO: `/v3/services/image` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:08.113092 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6931cd-4ee3-4db5-ba78-4173a4278c83 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:08.115811 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6931cd-4ee3-4db5-ba78-4173a4278c83 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:08.138833 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-cf6931cd-4ee3-4db5-ba78-4173a4278c83 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 21 18:14:08.138833 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:14:08.138833 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:14:08.138833 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:14:08.138833 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:14:08.138833 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:14:08.138833 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:14:08.138833 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:14:08.138833 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 21 18:14:08.138833 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:14:08.138833 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:14:08.138833 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:14:08.138833 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:14:08.138833 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:14:08.138833 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:14:08.138833 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:14:08.138833 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:14:08.140568 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:14:08.140568 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:14:08.140568 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:14:08.140568 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:14:08.140568 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:14:08.140568 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:14:08.140568 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:14:08.140568 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:14:08.140568 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:14:08.140568 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:14:08.140568 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:14:08.140568 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:14:08.140568 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:14:08.140568 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:14:08.140568 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:14:08.140568 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:14:08.149527 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:14:08.149527 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:14:08.149527 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:14:08.149527 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:14:08.149527 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:14:08.149527 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:14:08.149527 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:14:08.149527 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:14:08.149527 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:14:08.149527 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:14:08.149527 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:14:08.149527 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:14:08.149527 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 21 18:14:08.149527 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:14:08.149527 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 497/1005] 10.222.0.87 () {56 vars in 1105 bytes} [Tue Jun 21 18:14:07 2022] GET /identity/v3/services/image => generated 86 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:14:08.151281 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263839c8-1b69-4c55-9cf7-121d59b3c9c3 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:08.182208 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:08.386941 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263839c8-1b69-4c55-9cf7-121d59b3c9c3 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:08.467414 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce0e49-826f-4f92-a92d-e13a18adf82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:08.468939 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce0e49-826f-4f92-a92d-e13a18adf82b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:08.469437 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce0e49-826f-4f92-a92d-e13a18adf82b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:08.469963 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce0e49-826f-4f92-a92d-e13a18adf82b None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:08.471391 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ce0e49-826f-4f92-a92d-e13a18adf82b None admin] RBAC: Adding query filter params (name=image) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:08.473520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ce0e49-826f-4f92-a92d-e13a18adf82b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:08.474695 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ce0e49-826f-4f92-a92d-e13a18adf82b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:08.485477 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 509/1006] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:14:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:14:08.494659 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6931cd-4ee3-4db5-ba78-4173a4278c83 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:08.641611 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:08.731542 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6931cd-4ee3-4db5-ba78-4173a4278c83 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:08.813814 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8ddd43-a9bc-4a2a-89c4-d6f027bd282e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:08.813814 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ddd43-a9bc-4a2a-89c4-d6f027bd282e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:08.813814 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ddd43-a9bc-4a2a-89c4-d6f027bd282e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:08.814395 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ddd43-a9bc-4a2a-89c4-d6f027bd282e None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:08.815756 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8ddd43-a9bc-4a2a-89c4-d6f027bd282e None admin] RBAC: Adding query filter params (type=image) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:08.816279 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8ddd43-a9bc-4a2a-89c4-d6f027bd282e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:08.818989 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8ddd43-a9bc-4a2a-89c4-d6f027bd282e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:08.831083 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 498/1007] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:14:08 2022] GET /identity/v3/services?type=image => generated 122 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:14:09.027964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e16013-3640-4918-b6c6-4c98064607f0 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:09.029515 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e16013-3640-4918-b6c6-4c98064607f0 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:09.029515 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e16013-3640-4918-b6c6-4c98064607f0 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:09.030780 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 510/1008] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:14:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:14:09.045984 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1998b-f77a-446a-9f95-633b5bbebde6 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:09.046473 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1998b-f77a-446a-9f95-633b5bbebde6 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:09.046993 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1998b-f77a-446a-9f95-633b5bbebde6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:09.087510 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-2dd1998b-f77a-446a-9f95-633b5bbebde6 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:14:09.089831 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2dd1998b-f77a-446a-9f95-633b5bbebde6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:14:09.251679 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 499/1009] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:14:09 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:14:09.292692 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e16013-3640-4918-b6c6-4c98064607f0 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:09.463589 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:09.516487 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e16013-3640-4918-b6c6-4c98064607f0 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:09.619979 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02c2645-95dc-4155-a9d1-d094ac982d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:09.621398 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c2645-95dc-4155-a9d1-d094ac982d48 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:09.621919 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c2645-95dc-4155-a9d1-d094ac982d48 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:09.622430 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c2645-95dc-4155-a9d1-d094ac982d48 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:09.623611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02c2645-95dc-4155-a9d1-d094ac982d48 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:09.625383 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02c2645-95dc-4155-a9d1-d094ac982d48 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:09.643326 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-b02c2645-95dc-4155-a9d1-d094ac982d48 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 21 18:14:09.643326 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:14:09.643326 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:14:09.643326 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:14:09.643326 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:14:09.643326 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:14:09.643326 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:14:09.643326 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:14:09.643326 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 21 18:14:09.643326 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:14:09.643326 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:14:09.643326 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:14:09.643326 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:14:09.643326 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:14:09.643326 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:14:09.643326 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:14:09.643326 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:14:09.645849 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:14:09.645849 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:14:09.645849 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:14:09.645849 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:14:09.645849 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:14:09.645849 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:14:09.645849 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:14:09.645849 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:14:09.645849 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:14:09.645849 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:14:09.645849 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:14:09.645849 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:14:09.645849 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:14:09.645849 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:14:09.645849 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:14:09.645849 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:14:09.655184 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:14:09.655184 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:14:09.655184 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:14:09.655184 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:14:09.655184 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:14:09.655184 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:14:09.655184 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:14:09.655184 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:14:09.655184 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:14:09.655184 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:14:09.655184 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:14:09.655184 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:14:09.655184 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 21 18:14:09.655184 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:14:09.655184 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 511/1010] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:14:09 2022] GET /identity/v3/services/compute => generated 88 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:14:09.658884 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd1998b-f77a-446a-9f95-633b5bbebde6 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:09.806664 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:09.845048 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191ae92-8e92-4996-ad13-53a86da18ee3 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:09.845554 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191ae92-8e92-4996-ad13-53a86da18ee3 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:09.846077 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191ae92-8e92-4996-ad13-53a86da18ee3 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:09.849351 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 512/1011] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:14:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:14:09.859157 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da63180-011a-431c-9112-7bced07be268 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:09.859407 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da63180-011a-431c-9112-7bced07be268 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:09.859738 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da63180-011a-431c-9112-7bced07be268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:09.880867 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-9da63180-011a-431c-9112-7bced07be268 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:14:09.882530 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-9da63180-011a-431c-9112-7bced07be268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:14:09.895546 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd1998b-f77a-446a-9f95-633b5bbebde6 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:09.977092 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213cfa85-e553-4f49-8275-475c65fd4dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:09.977092 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213cfa85-e553-4f49-8275-475c65fd4dfd None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:09.977092 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213cfa85-e553-4f49-8275-475c65fd4dfd None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:09.978663 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213cfa85-e553-4f49-8275-475c65fd4dfd None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:09.979039 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213cfa85-e553-4f49-8275-475c65fd4dfd None admin] RBAC: Adding query filter params (name=compute) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:09.979674 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213cfa85-e553-4f49-8275-475c65fd4dfd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:09.981749 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213cfa85-e553-4f49-8275-475c65fd4dfd None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:10.000934 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 500/1012] 10.222.0.87 () {56 vars in 1112 bytes} [Tue Jun 21 18:14:09 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:14:10.002984 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213cfa85-e553-4f49-8275-475c65fd4dfd None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:10.030704 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 513/1013] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:14:09 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:14:10.064384 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da63180-011a-431c-9112-7bced07be268 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:10.197587 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:10.207123 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213cfa85-e553-4f49-8275-475c65fd4dfd None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:10.247690 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:10.255858 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da63180-011a-431c-9112-7bced07be268 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:10.279038 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7940fe-b650-45b4-9289-85f3831ecf1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:10.280349 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7940fe-b650-45b4-9289-85f3831ecf1c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:10.280876 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7940fe-b650-45b4-9289-85f3831ecf1c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:10.281389 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7940fe-b650-45b4-9289-85f3831ecf1c None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:10.282815 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7940fe-b650-45b4-9289-85f3831ecf1c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:10.283363 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7940fe-b650-45b4-9289-85f3831ecf1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:10.285428 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7940fe-b650-45b4-9289-85f3831ecf1c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:10.291924 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 501/1014] 10.222.0.87 () {56 vars in 1112 bytes} [Tue Jun 21 18:14:10 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:14:10.299703 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7940fe-b650-45b4-9289-85f3831ecf1c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:10.327010 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b274654b-66cc-47c8-81c1-3df775db1c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:10.328176 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274654b-66cc-47c8-81c1-3df775db1c7b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:10.328580 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274654b-66cc-47c8-81c1-3df775db1c7b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:10.328999 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274654b-66cc-47c8-81c1-3df775db1c7b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:10.331031 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b274654b-66cc-47c8-81c1-3df775db1c7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:10.333821 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b274654b-66cc-47c8-81c1-3df775db1c7b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:10.337745 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 514/1015] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:14:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:14:10.347106 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b274654b-66cc-47c8-81c1-3df775db1c7b None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:10.531301 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7940fe-b650-45b4-9289-85f3831ecf1c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:10.531595 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b274654b-66cc-47c8-81c1-3df775db1c7b None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:10.600480 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1b2629-1f9e-4887-847c-1fd4f92b852e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:10.601477 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b2629-1f9e-4887-847c-1fd4f92b852e None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:10.602001 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b2629-1f9e-4887-847c-1fd4f92b852e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:10.602433 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b2629-1f9e-4887-847c-1fd4f92b852e None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:10.603701 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1b2629-1f9e-4887-847c-1fd4f92b852e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:10.607225 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1b2629-1f9e-4887-847c-1fd4f92b852e None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:10.616385 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed077d-772f-4bbb-a276-3575fa36f050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:10.621333 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed077d-772f-4bbb-a276-3575fa36f050 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:10.621333 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed077d-772f-4bbb-a276-3575fa36f050 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:10.621333 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed077d-772f-4bbb-a276-3575fa36f050 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:10.625695 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:10.626955 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbed077d-772f-4bbb-a276-3575fa36f050 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:10.630523 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbed077d-772f-4bbb-a276-3575fa36f050 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:10.672803 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 502/1016] 10.222.0.87 () {60 vars in 1144 bytes} [Tue Jun 21 18:14:10 2022] POST /identity/v3/endpoints => generated 334 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:14:10.677708 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 515/1017] 10.222.0.87 () {60 vars in 1141 bytes} [Tue Jun 21 18:14:10 2022] POST /identity/v3/projects => generated 311 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:14:12.214705 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8360aa-5378-43fa-9867-79dbde96c31f None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:12.215589 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8360aa-5378-43fa-9867-79dbde96c31f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:12.215589 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8360aa-5378-43fa-9867-79dbde96c31f None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:12.217695 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 503/1018] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:14:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:14:12.225266 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090476d9-be46-426f-b62c-8716683de2bf None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:12.226146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090476d9-be46-426f-b62c-8716683de2bf None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:12.226286 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090476d9-be46-426f-b62c-8716683de2bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:12.265973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-090476d9-be46-426f-b62c-8716683de2bf None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:14:12.268583 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-090476d9-be46-426f-b62c-8716683de2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:14:12.469424 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 516/1019] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:14:12 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:14:12.502637 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8360aa-5378-43fa-9867-79dbde96c31f None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:12.612866 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:12.755579 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8360aa-5378-43fa-9867-79dbde96c31f None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:12.849553 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826d035f-e37c-4906-8fb8-cf4a7de904d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:12.852061 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d035f-e37c-4906-8fb8-cf4a7de904d0 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:12.852535 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d035f-e37c-4906-8fb8-cf4a7de904d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:12.852957 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d035f-e37c-4906-8fb8-cf4a7de904d0 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:12.854438 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826d035f-e37c-4906-8fb8-cf4a7de904d0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:12.856142 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826d035f-e37c-4906-8fb8-cf4a7de904d0 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:12.883891 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 504/1020] 10.222.0.87 () {60 vars in 1141 bytes} [Tue Jun 21 18:14:12 2022] POST /identity/v3/services => generated 242 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:14:14.431922 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700177ce-ec9e-4c38-9793-d2f6a67d3d75 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:14.432816 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700177ce-ec9e-4c38-9793-d2f6a67d3d75 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:14.432953 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700177ce-ec9e-4c38-9793-d2f6a67d3d75 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:14.435626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 517/1021] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:14:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:14:14.445756 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40ebe8-217e-4ab9-b876-475dcb1924c4 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:14.446615 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40ebe8-217e-4ab9-b876-475dcb1924c4 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:14.446778 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40ebe8-217e-4ab9-b876-475dcb1924c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:14.475257 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f7394-14f1-4b3d-863a-168bd087593c None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:14.475752 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f7394-14f1-4b3d-863a-168bd087593c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:14.476466 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f7394-14f1-4b3d-863a-168bd087593c None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:14.479977 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 518/1022] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:14:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:14:14.481488 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-de40ebe8-217e-4ab9-b876-475dcb1924c4 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:14:14.483756 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-de40ebe8-217e-4ab9-b876-475dcb1924c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:14:14.489446 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80984aa3-34a6-4c1c-93a7-ac30744e2c7e None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:14.489997 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80984aa3-34a6-4c1c-93a7-ac30744e2c7e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:14.490474 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80984aa3-34a6-4c1c-93a7-ac30744e2c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:14.524561 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-80984aa3-34a6-4c1c-93a7-ac30744e2c7e None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:14:14.526591 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-80984aa3-34a6-4c1c-93a7-ac30744e2c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:14:14.666606 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 505/1023] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:14:14 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:14:14.697915 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 519/1024] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:14:14 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:14:14.712464 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de40ebe8-217e-4ab9-b876-475dcb1924c4 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:14.735282 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80984aa3-34a6-4c1c-93a7-ac30744e2c7e None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:14.788315 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:14.844139 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:14.943793 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de40ebe8-217e-4ab9-b876-475dcb1924c4 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:14.951699 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80984aa3-34a6-4c1c-93a7-ac30744e2c7e None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:15.008467 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef36708b-1bff-41f2-b15f-194a67bd889f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:15.009550 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36708b-1bff-41f2-b15f-194a67bd889f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:15.010007 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36708b-1bff-41f2-b15f-194a67bd889f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:15.010483 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36708b-1bff-41f2-b15f-194a67bd889f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:15.025261 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db7dc4-888c-4f36-9014-438697ae36d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:15.026368 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db7dc4-888c-4f36-9014-438697ae36d6 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:15.026735 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db7dc4-888c-4f36-9014-438697ae36d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:15.027035 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db7dc4-888c-4f36-9014-438697ae36d6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:15.028623 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37db7dc4-888c-4f36-9014-438697ae36d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:15.030672 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37db7dc4-888c-4f36-9014-438697ae36d6 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:15.033485 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 520/1025] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:14:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:14:15.038144 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef36708b-1bff-41f2-b15f-194a67bd889f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:14:15.042919 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db7dc4-888c-4f36-9014-438697ae36d6 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:15.048168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-ef36708b-1bff-41f2-b15f-194a67bd889f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 18:14:15.048168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:14:15.048168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:14:15.048168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:14:15.048168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:14:15.048168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:14:15.048168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:14:15.048168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:14:15.048168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:14:15.048168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:14:15.048168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:14:15.048168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:14:15.048168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:14:15.048168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:14:15.048168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:14:15.048168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:14:15.050229 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:14:15.050229 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:14:15.050229 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:14:15.050229 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:14:15.050229 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:14:15.050229 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:14:15.050229 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:14:15.050229 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:14:15.050229 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:14:15.050229 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:14:15.050229 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:14:15.050229 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:14:15.050229 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:14:15.050229 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:14:15.050229 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:14:15.050229 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:14:15.051715 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:14:15.051715 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:14:15.051715 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:14:15.051715 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:14:15.051715 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:14:15.051715 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:14:15.051715 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:14:15.051715 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:14:15.051715 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:14:15.051715 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:14:15.051715 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 18:14:15.051715 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:14:15.051715 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 506/1026] 10.222.0.87 () {56 vars in 1120 bytes} [Tue Jun 21 18:14:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:14:15.060319 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef36708b-1bff-41f2-b15f-194a67bd889f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:15.264497 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:15.274398 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db7dc4-888c-4f36-9014-438697ae36d6 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:15.279968 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef36708b-1bff-41f2-b15f-194a67bd889f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:15.310765 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:15.330169 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3077e516-f9d3-4667-8db3-f82c2a313505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:15.331193 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077e516-f9d3-4667-8db3-f82c2a313505 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:15.331661 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077e516-f9d3-4667-8db3-f82c2a313505 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:15.332079 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077e516-f9d3-4667-8db3-f82c2a313505 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:15.333021 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3077e516-f9d3-4667-8db3-f82c2a313505 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:15.333484 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3077e516-f9d3-4667-8db3-f82c2a313505 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:15.335468 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3077e516-f9d3-4667-8db3-f82c2a313505 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:15.352776 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 507/1027] 10.222.0.87 () {56 vars in 1115 bytes} [Tue Jun 21 18:14:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:14:15.353030 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23608f1d-7a7b-4142-aeb0-cbbce9c5d56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:15.353030 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23608f1d-7a7b-4142-aeb0-cbbce9c5d56b None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:15.353030 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23608f1d-7a7b-4142-aeb0-cbbce9c5d56b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:15.353982 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23608f1d-7a7b-4142-aeb0-cbbce9c5d56b None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:15.354533 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3077e516-f9d3-4667-8db3-f82c2a313505 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:15.354805 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23608f1d-7a7b-4142-aeb0-cbbce9c5d56b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:15.357288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23608f1d-7a7b-4142-aeb0-cbbce9c5d56b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:15.414231 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 521/1028] 10.222.0.87 () {60 vars in 1132 bytes} [Tue Jun 21 18:14:15 2022] POST /identity/v3/users => generated 285 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:14:15.560863 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3077e516-f9d3-4667-8db3-f82c2a313505 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:15.647593 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e823e6a-93c9-4fa6-898f-e7a077bb840d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:15.648997 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e823e6a-93c9-4fa6-898f-e7a077bb840d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:15.649496 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e823e6a-93c9-4fa6-898f-e7a077bb840d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:15.650049 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e823e6a-93c9-4fa6-898f-e7a077bb840d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:15.672368 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e823e6a-93c9-4fa6-898f-e7a077bb840d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:15.675286 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e823e6a-93c9-4fa6-898f-e7a077bb840d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:15.711718 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-6e823e6a-93c9-4fa6-898f-e7a077bb840d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:14:15.711718 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:14:15.711718 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:14:15.711718 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:14:15.711718 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:14:15.711718 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:14:15.711718 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:14:15.711718 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:14:15.711718 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:14:15.711718 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:14:15.711718 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:14:15.711718 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:14:15.711718 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:14:15.711718 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:14:15.711718 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:14:15.713535 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:14:15.713535 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:14:15.713535 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:14:15.713535 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:14:15.713535 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:14:15.713535 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:14:15.713535 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:14:15.713535 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:14:15.713535 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:14:15.713535 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:14:15.713535 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:14:15.713535 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:14:15.713535 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:14:15.713535 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:14:15.713535 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:14:15.713535 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:14:15.713535 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:14:15.715411 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:14:15.715411 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:14:15.715411 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:14:15.715411 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:14:15.718182 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 508/1029] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:14:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:14:15.723102 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23608f1d-7a7b-4142-aeb0-cbbce9c5d56b None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:15.967859 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23608f1d-7a7b-4142-aeb0-cbbce9c5d56b None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:16.050549 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d27768-4f3d-461a-957b-e927093d2e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:16.052110 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d27768-4f3d-461a-957b-e927093d2e90 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:16.052805 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d27768-4f3d-461a-957b-e927093d2e90 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:16.053506 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d27768-4f3d-461a-957b-e927093d2e90 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:16.055536 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d27768-4f3d-461a-957b-e927093d2e90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:16.056416 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d27768-4f3d-461a-957b-e927093d2e90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:16.059441 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d27768-4f3d-461a-957b-e927093d2e90 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:16.092950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 522/1030] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:14:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:14:16.097720 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e823e6a-93c9-4fa6-898f-e7a077bb840d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:16.105064 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:16.319904 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e823e6a-93c9-4fa6-898f-e7a077bb840d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:16.404853 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbde37c5-4e2b-4a86-b37d-de4f8e6d8f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:16.406186 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde37c5-4e2b-4a86-b37d-de4f8e6d8f6c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:16.406702 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde37c5-4e2b-4a86-b37d-de4f8e6d8f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:16.407196 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde37c5-4e2b-4a86-b37d-de4f8e6d8f6c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:16.423947 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbde37c5-4e2b-4a86-b37d-de4f8e6d8f6c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:16.427147 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbde37c5-4e2b-4a86-b37d-de4f8e6d8f6c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:16.449503 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c215184d-b715-43c6-a2fb-a53e0284942d None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:16.450081 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c215184d-b715-43c6-a2fb-a53e0284942d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:16.450560 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c215184d-b715-43c6-a2fb-a53e0284942d None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:16.453554 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 523/1031] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:14:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:14:16.454321 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-fbde37c5-4e2b-4a86-b37d-de4f8e6d8f6c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:14:16.454321 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:14:16.454321 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:14:16.454321 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:14:16.454321 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:14:16.454321 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:14:16.454321 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:14:16.454321 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:14:16.454321 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:14:16.454321 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:14:16.454321 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:14:16.454321 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:14:16.454321 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:14:16.454321 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:14:16.454321 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:14:16.454321 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:14:16.456481 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:14:16.456481 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:14:16.456481 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:14:16.456481 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:14:16.456481 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:14:16.456481 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:14:16.456481 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:14:16.456481 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:14:16.456481 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:14:16.456481 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:14:16.456481 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:14:16.456481 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:14:16.456481 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:14:16.456481 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:14:16.456481 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:14:16.456481 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:14:16.456481 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:14:16.456481 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:14:16.459260 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:14:16.459260 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:14:16.459260 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:14:16.459260 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:14:16.459260 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:14:16.459260 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:14:16.459260 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:14:16.459260 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:14:16.459260 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:14:16.459260 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:14:16.459260 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 509/1032] 10.222.0.87 () {56 vars in 1093 bytes} [Tue Jun 21 18:14:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:14:16.464858 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6ac745-02e1-45a2-8b24-1c8471c9ccb3 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:16.465345 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6ac745-02e1-45a2-8b24-1c8471c9ccb3 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:16.465894 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6ac745-02e1-45a2-8b24-1c8471c9ccb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:16.468214 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbde37c5-4e2b-4a86-b37d-de4f8e6d8f6c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:16.503700 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-9a6ac745-02e1-45a2-8b24-1c8471c9ccb3 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:14:16.505886 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-9a6ac745-02e1-45a2-8b24-1c8471c9ccb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:14:16.569751 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:16.604830 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:16.686292 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 524/1033] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:14:16 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:14:16.696905 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbde37c5-4e2b-4a86-b37d-de4f8e6d8f6c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:16.719666 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6ac745-02e1-45a2-8b24-1c8471c9ccb3 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:16.767370 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a1a70-74b7-43ab-9429-beb7537ef78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:16.768544 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a1a70-74b7-43ab-9429-beb7537ef78d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:16.769047 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a1a70-74b7-43ab-9429-beb7537ef78d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:16.769486 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a1a70-74b7-43ab-9429-beb7537ef78d None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:16.771150 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973a1a70-74b7-43ab-9429-beb7537ef78d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:16.771681 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973a1a70-74b7-43ab-9429-beb7537ef78d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:16.774449 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973a1a70-74b7-43ab-9429-beb7537ef78d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:16.815751 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 510/1034] 10.222.0.87 () {56 vars in 1133 bytes} [Tue Jun 21 18:14:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:14:16.825545 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a1a70-74b7-43ab-9429-beb7537ef78d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:16.941989 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6ac745-02e1-45a2-8b24-1c8471c9ccb3 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:17.010277 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e559b796-c972-4e89-99a9-7de93a0f088e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:17.011637 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559b796-c972-4e89-99a9-7de93a0f088e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:17.012138 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559b796-c972-4e89-99a9-7de93a0f088e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:17.012689 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559b796-c972-4e89-99a9-7de93a0f088e None admin] PATH_INFO: `/v3/services/image` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:17.013905 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e559b796-c972-4e89-99a9-7de93a0f088e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:17.016506 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e559b796-c972-4e89-99a9-7de93a0f088e None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:17.036930 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a1a70-74b7-43ab-9429-beb7537ef78d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:17.038552 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-e559b796-c972-4e89-99a9-7de93a0f088e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 21 18:14:17.038552 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:14:17.038552 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:14:17.038552 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:14:17.038552 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:14:17.038552 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:14:17.038552 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:14:17.038552 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:14:17.038552 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 21 18:14:17.038552 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:14:17.038552 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:14:17.038552 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:14:17.038552 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:14:17.038552 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:14:17.038552 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:14:17.038552 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:14:17.038552 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:14:17.040425 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:14:17.040425 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:14:17.040425 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:14:17.040425 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:14:17.040425 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:14:17.040425 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:14:17.040425 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:14:17.040425 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:14:17.040425 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:14:17.040425 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:14:17.040425 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:14:17.040425 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:14:17.040425 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:14:17.040425 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:14:17.040425 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:14:17.040425 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:14:17.042010 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:14:17.042010 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:14:17.042010 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:14:17.042010 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:14:17.042010 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:14:17.042010 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:14:17.042010 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:14:17.042010 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:14:17.042010 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:14:17.042010 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:14:17.042010 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:14:17.042010 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:14:17.042010 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 21 18:14:17.042010 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:14:17.042010 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 525/1035] 10.222.0.87 () {56 vars in 1105 bytes} [Tue Jun 21 18:14:16 2022] GET /identity/v3/services/image => generated 86 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:14:17.050814 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e559b796-c972-4e89-99a9-7de93a0f088e None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:17.090522 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:17.106341 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:17.111627 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb5c68-e7d1-4ba4-8e4c-cdc9c508ca97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:17.112624 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb5c68-e7d1-4ba4-8e4c-cdc9c508ca97 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:17.112923 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb5c68-e7d1-4ba4-8e4c-cdc9c508ca97 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:17.113208 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb5c68-e7d1-4ba4-8e4c-cdc9c508ca97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:17.255053 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cb5c68-e7d1-4ba4-8e4c-cdc9c508ca97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:17.257773 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cb5c68-e7d1-4ba4-8e4c-cdc9c508ca97 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:17.278877 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e559b796-c972-4e89-99a9-7de93a0f088e None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:17.357532 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee7a4d6-b42a-47a7-a851-926091894820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:17.358777 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee7a4d6-b42a-47a7-a851-926091894820 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:17.359364 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee7a4d6-b42a-47a7-a851-926091894820 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:17.359790 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee7a4d6-b42a-47a7-a851-926091894820 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:17.361257 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee7a4d6-b42a-47a7-a851-926091894820 None admin] RBAC: Adding query filter params (name=image) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:17.361859 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee7a4d6-b42a-47a7-a851-926091894820 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:17.364579 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee7a4d6-b42a-47a7-a851-926091894820 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:17.375202 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 526/1036] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:14:17 2022] GET /identity/v3/services?name=image => generated 122 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:14:17.385984 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee7a4d6-b42a-47a7-a851-926091894820 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:17.479192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 511/1037] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:14:16 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 655 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 21 18:14:17.489894 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb5c68-e7d1-4ba4-8e4c-cdc9c508ca97 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:17.548051 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:17.603557 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:17.627672 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee7a4d6-b42a-47a7-a851-926091894820 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:17.713137 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dbff28-721a-4249-b2d3-a26e30cf7e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:17.714383 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dbff28-721a-4249-b2d3-a26e30cf7e41 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:17.714862 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dbff28-721a-4249-b2d3-a26e30cf7e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:17.715426 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dbff28-721a-4249-b2d3-a26e30cf7e41 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:17.717227 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dbff28-721a-4249-b2d3-a26e30cf7e41 None admin] RBAC: Adding query filter params (type=image) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:17.717817 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dbff28-721a-4249-b2d3-a26e30cf7e41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:17.720597 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dbff28-721a-4249-b2d3-a26e30cf7e41 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:17.737678 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 527/1038] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:14:17 2022] GET /identity/v3/services?type=image => generated 350 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:14:17.737959 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb5c68-e7d1-4ba4-8e4c-cdc9c508ca97 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:17.739225 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dbff28-721a-4249-b2d3-a26e30cf7e41 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:17.813980 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db047fa2-5dae-4b02-b4c0-a7cff92703b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:17.815331 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db047fa2-5dae-4b02-b4c0-a7cff92703b3 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:17.815863 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db047fa2-5dae-4b02-b4c0-a7cff92703b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:17.816344 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db047fa2-5dae-4b02-b4c0-a7cff92703b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:17.846480 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db047fa2-5dae-4b02-b4c0-a7cff92703b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:17.850362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db047fa2-5dae-4b02-b4c0-a7cff92703b3 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:17.886760 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-db047fa2-5dae-4b02-b4c0-a7cff92703b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:14:17.886760 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:14:17.886760 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:14:17.886760 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:14:17.886760 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:14:17.886760 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:14:17.886760 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:14:17.886760 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:14:17.886760 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:14:17.886760 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:14:17.886760 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:14:17.886760 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:14:17.886760 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:14:17.886760 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:14:17.886760 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:14:17.887914 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:14:17.887914 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:14:17.887914 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:14:17.887914 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:14:17.887914 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:14:17.887914 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:14:17.887914 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:14:17.887914 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:14:17.887914 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:14:17.887914 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:14:17.887914 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:14:17.887914 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:14:17.887914 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:14:17.887914 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:14:17.887914 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:14:17.887914 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:14:17.887914 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:14:17.889392 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:14:17.889392 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:14:17.889392 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:14:17.889392 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:14:17.889681 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 512/1039] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:14:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 401 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:14:17.899450 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db047fa2-5dae-4b02-b4c0-a7cff92703b3 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:17.971613 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dbff28-721a-4249-b2d3-a26e30cf7e41 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:18.035984 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d8e49d-8a1a-4590-9d1f-c56b1f5c6df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:18.037190 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8e49d-8a1a-4590-9d1f-c56b1f5c6df1 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:18.037706 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8e49d-8a1a-4590-9d1f-c56b1f5c6df1 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:18.038207 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8e49d-8a1a-4590-9d1f-c56b1f5c6df1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:18.040054 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d8e49d-8a1a-4590-9d1f-c56b1f5c6df1 None admin] RBAC: Adding query filter params (interface=public, service_id=e1ebc23caedf495e94a71b2b2bbdcbfd, region_id=RegionOne) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:18.040642 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d8e49d-8a1a-4590-9d1f-c56b1f5c6df1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:18.043806 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d8e49d-8a1a-4590-9d1f-c56b1f5c6df1 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:18.053545 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:18.061977 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 528/1040] 10.222.0.87 () {56 vars in 1251 bytes} [Tue Jun 21 18:14:17 2022] GET /identity/v3/endpoints?service_id=e1ebc23caedf495e94a71b2b2bbdcbfd&interface=public®ion_id=RegionOne => generated 194 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:14:18.065120 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d8e49d-8a1a-4590-9d1f-c56b1f5c6df1 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:18.109012 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:18.128675 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db047fa2-5dae-4b02-b4c0-a7cff92703b3 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:18.216957 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f66968-2131-42b8-98c4-5b243bc1adf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:18.216957 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f66968-2131-42b8-98c4-5b243bc1adf9 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:18.216957 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f66968-2131-42b8-98c4-5b243bc1adf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:18.216957 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f66968-2131-42b8-98c4-5b243bc1adf9 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:18.218224 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f66968-2131-42b8-98c4-5b243bc1adf9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:18.218980 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f66968-2131-42b8-98c4-5b243bc1adf9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:18.222888 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f66968-2131-42b8-98c4-5b243bc1adf9 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:18.254315 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 513/1041] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:14:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:14:18.259954 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f66968-2131-42b8-98c4-5b243bc1adf9 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:18.314220 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d8e49d-8a1a-4590-9d1f-c56b1f5c6df1 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:18.393259 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259c6b78-b06a-4b17-b04b-2aa33ea93b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:18.394352 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c6b78-b06a-4b17-b04b-2aa33ea93b68 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:18.394834 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c6b78-b06a-4b17-b04b-2aa33ea93b68 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:18.395523 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c6b78-b06a-4b17-b04b-2aa33ea93b68 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:18.397060 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259c6b78-b06a-4b17-b04b-2aa33ea93b68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:18.399616 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259c6b78-b06a-4b17-b04b-2aa33ea93b68 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:18.415485 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 529/1042] 10.222.0.87 () {56 vars in 1087 bytes} [Tue Jun 21 18:14:18 2022] GET /identity/v3/services => generated 1957 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:14:18.513101 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f66968-2131-42b8-98c4-5b243bc1adf9 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:18.606492 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbce2cd3-abf4-4454-84c3-1d1f34095a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:18.609079 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce2cd3-abf4-4454-84c3-1d1f34095a60 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:18.609979 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce2cd3-abf4-4454-84c3-1d1f34095a60 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:18.610912 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce2cd3-abf4-4454-84c3-1d1f34095a60 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:18.616362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:18.631682 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbce2cd3-abf4-4454-84c3-1d1f34095a60 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:18.636445 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbce2cd3-abf4-4454-84c3-1d1f34095a60 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:18.668164 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-bbce2cd3-abf4-4454-84c3-1d1f34095a60 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:14:18.668164 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:14:18.668164 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:14:18.668164 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:14:18.668164 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:14:18.668164 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:14:18.668164 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:14:18.668164 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:14:18.668164 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:14:18.668164 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:14:18.668164 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:14:18.668164 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:14:18.668164 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:14:18.668164 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:14:18.668164 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:14:18.669936 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:14:18.669936 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:14:18.669936 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:14:18.669936 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:14:18.669936 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:14:18.669936 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:14:18.669936 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:14:18.669936 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:14:18.669936 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:14:18.669936 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:14:18.669936 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:14:18.669936 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:14:18.669936 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:14:18.669936 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:14:18.669936 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:14:18.669936 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:14:18.669936 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:14:18.671416 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:14:18.671416 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:14:18.671416 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:14:18.671416 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:14:18.671416 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:14:18.671416 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:14:18.671416 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:14:18.671416 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:14:18.677280 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 514/1043] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:14:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 418 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:14:18.682042 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259c6b78-b06a-4b17-b04b-2aa33ea93b68 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:18.750850 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba164d9e-f5dc-4639-aa9c-52d9831f2e48 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:18.751457 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba164d9e-f5dc-4639-aa9c-52d9831f2e48 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:18.751801 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba164d9e-f5dc-4639-aa9c-52d9831f2e48 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:18.754239 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 515/1044] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:14:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:14:18.764721 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc7189-6763-45e8-b1a5-489a849195fe None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:18.765230 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc7189-6763-45e8-b1a5-489a849195fe None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:18.765848 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc7189-6763-45e8-b1a5-489a849195fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:18.801074 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-03bc7189-6763-45e8-b1a5-489a849195fe None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:14:18.803381 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-03bc7189-6763-45e8-b1a5-489a849195fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:14:18.900298 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:18.962432 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259c6b78-b06a-4b17-b04b-2aa33ea93b68 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:18.970587 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 516/1045] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:14:18 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:14:19.008174 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc7189-6763-45e8-b1a5-489a849195fe None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:19.051925 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b1aeb4-61ff-4177-9e29-d963d91c9a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:19.052873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b1aeb4-61ff-4177-9e29-d963d91c9a05 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:19.053345 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b1aeb4-61ff-4177-9e29-d963d91c9a05 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:19.053821 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b1aeb4-61ff-4177-9e29-d963d91c9a05 None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:19.055327 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b1aeb4-61ff-4177-9e29-d963d91c9a05 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:19.055829 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b1aeb4-61ff-4177-9e29-d963d91c9a05 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:19.058511 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b1aeb4-61ff-4177-9e29-d963d91c9a05 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:19.085685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 530/1046] 10.222.0.87 () {56 vars in 1148 bytes} [Tue Jun 21 18:14:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:14:19.089762 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b1aeb4-61ff-4177-9e29-d963d91c9a05 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:19.257057 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc7189-6763-45e8-b1a5-489a849195fe None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:19.315022 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b1aeb4-61ff-4177-9e29-d963d91c9a05 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:19.324783 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:19.362314 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6efa9e1-9abe-4e9a-80f3-5395a142ac71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:19.363902 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6efa9e1-9abe-4e9a-80f3-5395a142ac71 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:19.364426 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6efa9e1-9abe-4e9a-80f3-5395a142ac71 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:19.364889 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6efa9e1-9abe-4e9a-80f3-5395a142ac71 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:19.390709 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6efa9e1-9abe-4e9a-80f3-5395a142ac71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:14:19.405329 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced9d063-3280-4066-8ed5-6ba071686b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:19.407167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-d6efa9e1-9abe-4e9a-80f3-5395a142ac71 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:14:19.407167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:14:19.407167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:14:19.407167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:14:19.407167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:14:19.407167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:14:19.407167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:14:19.407167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:14:19.407167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:14:19.407167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:14:19.407167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:14:19.407167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:14:19.407167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:14:19.407167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:14:19.407167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:14:19.407167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:14:19.411274 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:14:19.411274 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:14:19.411274 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:14:19.411274 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:14:19.411274 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:14:19.411274 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:14:19.411274 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:14:19.411274 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:14:19.411274 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:14:19.411274 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:14:19.411274 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:14:19.411274 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:14:19.411274 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:14:19.411274 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:14:19.411274 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:14:19.411274 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:14:19.412657 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:14:19.412657 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:14:19.412657 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:14:19.412657 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:14:19.412657 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:14:19.412657 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:14:19.412657 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:14:19.412657 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:14:19.412657 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:14:19.412657 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:14:19.412657 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:14:19.412657 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:14:19.413978 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced9d063-3280-4066-8ed5-6ba071686b47 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:19.413978 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced9d063-3280-4066-8ed5-6ba071686b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:19.413978 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced9d063-3280-4066-8ed5-6ba071686b47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:19.413978 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced9d063-3280-4066-8ed5-6ba071686b47 None admin] RBAC: Adding query filter params (role.id=9fa867d83f3c406aabf51ddd8c7ed2aa, scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb, user.id=a1b3fa67905c4027b3656ac36e2b76c9) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:19.413978 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced9d063-3280-4066-8ed5-6ba071686b47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:19.414561 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced9d063-3280-4066-8ed5-6ba071686b47 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:19.421317 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 517/1047] 10.222.0.87 () {56 vars in 1096 bytes} [Tue Jun 21 18:14:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 411 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:14:19.421317 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6efa9e1-9abe-4e9a-80f3-5395a142ac71 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:19.441989 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 531/1048] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:14:19 2022] GET /identity/v3/role_assignments?user.id=a1b3fa67905c4027b3656ac36e2b76c9&scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb&role.id=9fa867d83f3c406aabf51ddd8c7ed2aa => generated 259 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:14:19.703406 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6efa9e1-9abe-4e9a-80f3-5395a142ac71 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:19.792029 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e88578-451c-4329-b27b-c69046744074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:19.793291 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e88578-451c-4329-b27b-c69046744074 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:19.793787 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e88578-451c-4329-b27b-c69046744074 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:19.794277 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e88578-451c-4329-b27b-c69046744074 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:19.795731 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e88578-451c-4329-b27b-c69046744074 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:19.796267 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e88578-451c-4329-b27b-c69046744074 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:19.800391 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e88578-451c-4329-b27b-c69046744074 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:19.822802 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 518/1049] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:14:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:14:19.832364 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced9d063-3280-4066-8ed5-6ba071686b47 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:19.851820 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:20.062766 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced9d063-3280-4066-8ed5-6ba071686b47 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:20.136635 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50edb6a7-2827-415c-9863-19481f3e6c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:20.138301 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50edb6a7-2827-415c-9863-19481f3e6c8f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:20.138784 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50edb6a7-2827-415c-9863-19481f3e6c8f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:20.139275 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50edb6a7-2827-415c-9863-19481f3e6c8f None admin] PATH_INFO: `/v3/users/02e645c4c6e14786b95e1548c19514ec` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:20.167675 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50edb6a7-2827-415c-9863-19481f3e6c8f None admin] RBAC: Authorizing `identity:get_user(user_id=02e645c4c6e14786b95e1548c19514ec)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:20.170992 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50edb6a7-2827-415c-9863-19481f3e6c8f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:20.174959 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 532/1050] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:14:19 2022] GET /identity/v3/users/02e645c4c6e14786b95e1548c19514ec => generated 285 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:14:20.184759 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e88578-451c-4329-b27b-c69046744074 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:20.286093 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:20.391291 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e88578-451c-4329-b27b-c69046744074 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:20.445167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002cb11-97e5-4d04-83ad-f9a66b23e815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:20.445931 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002cb11-97e5-4d04-83ad-f9a66b23e815 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:20.446218 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002cb11-97e5-4d04-83ad-f9a66b23e815 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:20.446498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002cb11-97e5-4d04-83ad-f9a66b23e815 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:20.550890 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b002cb11-97e5-4d04-83ad-f9a66b23e815 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:20.552807 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b002cb11-97e5-4d04-83ad-f9a66b23e815 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:20.624316 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:20.720795 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 519/1051] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:14:20 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:14:20.728281 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50edb6a7-2827-415c-9863-19481f3e6c8f None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:20.849387 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:20.924286 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50edb6a7-2827-415c-9863-19481f3e6c8f None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:20.998954 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84b88f-dc29-4ec6-9812-93e60f878ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:21.000446 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84b88f-dc29-4ec6-9812-93e60f878ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:21.001081 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84b88f-dc29-4ec6-9812-93e60f878ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:21.001470 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84b88f-dc29-4ec6-9812-93e60f878ac3 None admin] PATH_INFO: `/v3/projects/bd9bd2473fcd4546b4200678e8df9d66` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:21.004118 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84b88f-dc29-4ec6-9812-93e60f878ac3 None admin] RBAC: Authorizing `identity:get_project(project_id=bd9bd2473fcd4546b4200678e8df9d66)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:21.006949 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84b88f-dc29-4ec6-9812-93e60f878ac3 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:21.018115 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 533/1052] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:14:20 2022] GET /identity/v3/projects/bd9bd2473fcd4546b4200678e8df9d66 => generated 311 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:14:21.020756 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002cb11-97e5-4d04-83ad-f9a66b23e815 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:21.233252 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002cb11-97e5-4d04-83ad-f9a66b23e815 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:21.300217 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65553ac0-a74d-42ef-8277-3608308d4bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:21.301241 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65553ac0-a74d-42ef-8277-3608308d4bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:21.301775 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65553ac0-a74d-42ef-8277-3608308d4bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:21.302264 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65553ac0-a74d-42ef-8277-3608308d4bf7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:21.303846 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65553ac0-a74d-42ef-8277-3608308d4bf7 None admin] RBAC: Adding query filter params (role.id=98f0943a610f4a9e9f8de54f6e9f717d, scope.project.id=bd9bd2473fcd4546b4200678e8df9d66, user.id=02e645c4c6e14786b95e1548c19514ec) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:21.304413 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65553ac0-a74d-42ef-8277-3608308d4bf7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:21.307218 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65553ac0-a74d-42ef-8277-3608308d4bf7 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:21.319213 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 520/1053] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:14:21 2022] GET /identity/v3/role_assignments?user.id=02e645c4c6e14786b95e1548c19514ec&scope.project.id=bd9bd2473fcd4546b4200678e8df9d66&role.id=98f0943a610f4a9e9f8de54f6e9f717d => generated 259 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:14:22.043054 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdddc8fc-6fe6-475b-9fae-e0e39669c237 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:22.043833 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdddc8fc-6fe6-475b-9fae-e0e39669c237 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:22.043970 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdddc8fc-6fe6-475b-9fae-e0e39669c237 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:22.046845 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 534/1054] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:14:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:14:22.056466 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb14f504-e5bc-45de-982e-f599f5139730 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:22.057403 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb14f504-e5bc-45de-982e-f599f5139730 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:22.057543 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb14f504-e5bc-45de-982e-f599f5139730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:22.082055 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-bb14f504-e5bc-45de-982e-f599f5139730 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:14:22.083356 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-bb14f504-e5bc-45de-982e-f599f5139730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:14:22.086983 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:22.223964 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 521/1055] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:14:22 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:14:22.250601 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdddc8fc-6fe6-475b-9fae-e0e39669c237 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:22.398280 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdddc8fc-6fe6-475b-9fae-e0e39669c237 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:22.441308 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:22.451958 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf73624-dc98-44e1-89e2-3fe2908555b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:22.453299 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf73624-dc98-44e1-89e2-3fe2908555b9 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:22.453911 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf73624-dc98-44e1-89e2-3fe2908555b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:22.454469 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf73624-dc98-44e1-89e2-3fe2908555b9 None admin] PATH_INFO: `/v3/services/image` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:22.455722 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf73624-dc98-44e1-89e2-3fe2908555b9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:22.458418 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf73624-dc98-44e1-89e2-3fe2908555b9 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:22.479760 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-ccf73624-dc98-44e1-89e2-3fe2908555b9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 21 18:14:22.479760 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:14:22.479760 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:14:22.479760 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:14:22.479760 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:14:22.479760 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:14:22.479760 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:14:22.479760 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:14:22.479760 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 21 18:14:22.479760 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:14:22.479760 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:14:22.479760 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:14:22.479760 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:14:22.479760 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:14:22.479760 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:14:22.479760 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:14:22.479760 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:14:22.482196 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:14:22.482196 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:14:22.482196 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:14:22.482196 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:14:22.482196 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:14:22.482196 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:14:22.482196 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:14:22.482196 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:14:22.482196 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:14:22.482196 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:14:22.482196 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:14:22.482196 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:14:22.482196 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:14:22.482196 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:14:22.482196 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:14:22.482196 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:14:22.484068 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:14:22.484068 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:14:22.484068 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:14:22.484068 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:14:22.484068 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:14:22.484068 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:14:22.484068 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:14:22.484068 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:14:22.484068 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:14:22.484068 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:14:22.484068 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:14:22.484068 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:14:22.484068 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 21 18:14:22.484068 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:14:22.489029 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 535/1056] 10.222.0.87 () {56 vars in 1105 bytes} [Tue Jun 21 18:14:22 2022] GET /identity/v3/services/image => generated 86 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:14:22.490586 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb14f504-e5bc-45de-982e-f599f5139730 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:22.644889 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb14f504-e5bc-45de-982e-f599f5139730 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:22.718494 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a33444-0117-45f6-9e5c-7b10f22cf74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:22.719711 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a33444-0117-45f6-9e5c-7b10f22cf74a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:22.720205 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a33444-0117-45f6-9e5c-7b10f22cf74a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:22.720693 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a33444-0117-45f6-9e5c-7b10f22cf74a None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:22.722117 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a33444-0117-45f6-9e5c-7b10f22cf74a None admin] RBAC: Adding query filter params (name=image) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:22.722808 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a33444-0117-45f6-9e5c-7b10f22cf74a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:22.725340 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a33444-0117-45f6-9e5c-7b10f22cf74a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:22.732069 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1351cbf-24e1-412c-bb3a-84d4a8cb571a None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:22.732591 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1351cbf-24e1-412c-bb3a-84d4a8cb571a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:22.733696 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1351cbf-24e1-412c-bb3a-84d4a8cb571a None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:22.735682 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 536/1057] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:14:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:14:22.743716 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 522/1058] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:14:22 2022] GET /identity/v3/services?name=image => generated 122 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:14:22.743996 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58433f05-8bf4-43a3-bc18-56e23d5b10cf None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:22.743996 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58433f05-8bf4-43a3-bc18-56e23d5b10cf None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:22.743996 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58433f05-8bf4-43a3-bc18-56e23d5b10cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:22.747309 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a33444-0117-45f6-9e5c-7b10f22cf74a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:22.754477 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:22.767676 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-58433f05-8bf4-43a3-bc18-56e23d5b10cf None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:14:22.768689 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-58433f05-8bf4-43a3-bc18-56e23d5b10cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:14:22.931400 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 537/1059] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:14:22 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:14:22.972724 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58433f05-8bf4-43a3-bc18-56e23d5b10cf None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:22.986237 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a33444-0117-45f6-9e5c-7b10f22cf74a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:23.058447 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8214580c-ca3b-4794-89df-2a13cebfec17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:23.059854 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214580c-ca3b-4794-89df-2a13cebfec17 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:23.060443 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214580c-ca3b-4794-89df-2a13cebfec17 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:23.060936 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214580c-ca3b-4794-89df-2a13cebfec17 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:23.062364 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8214580c-ca3b-4794-89df-2a13cebfec17 None admin] RBAC: Adding query filter params (type=image) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:23.062906 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8214580c-ca3b-4794-89df-2a13cebfec17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:23.065678 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8214580c-ca3b-4794-89df-2a13cebfec17 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:23.075677 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 523/1060] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:14:22 2022] GET /identity/v3/services?type=image => generated 350 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:14:23.087907 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8214580c-ca3b-4794-89df-2a13cebfec17 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:23.181885 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58433f05-8bf4-43a3-bc18-56e23d5b10cf None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:23.230332 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:23.237159 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:23.257965 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e01a29-075a-436a-b4ca-098afee1c699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:23.259110 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e01a29-075a-436a-b4ca-098afee1c699 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:23.259538 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e01a29-075a-436a-b4ca-098afee1c699 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:23.259944 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e01a29-075a-436a-b4ca-098afee1c699 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:23.277358 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e01a29-075a-436a-b4ca-098afee1c699 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:14:23.291824 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-d6e01a29-075a-436a-b4ca-098afee1c699 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 18:14:23.291824 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:14:23.291824 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:14:23.291824 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:14:23.291824 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:14:23.291824 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:14:23.291824 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:14:23.291824 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:14:23.291824 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:14:23.291824 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:14:23.291824 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:14:23.291824 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:14:23.291824 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:14:23.291824 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:14:23.291824 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:14:23.291824 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:14:23.293380 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:14:23.293380 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:14:23.293380 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:14:23.293380 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:14:23.293380 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:14:23.293380 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:14:23.293380 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:14:23.293380 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:14:23.293380 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:14:23.293380 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:14:23.293380 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:14:23.293380 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:14:23.293380 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:14:23.293380 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:14:23.293380 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:14:23.293380 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:14:23.298928 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:14:23.298928 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:14:23.298928 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:14:23.298928 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:14:23.298928 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:14:23.298928 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:14:23.298928 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:14:23.298928 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:14:23.298928 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:14:23.298928 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:14:23.298928 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 18:14:23.298928 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:14:23.298928 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 538/1061] 10.222.0.87 () {56 vars in 1120 bytes} [Tue Jun 21 18:14:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:14:23.303673 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e01a29-075a-436a-b4ca-098afee1c699 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:23.314849 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8214580c-ca3b-4794-89df-2a13cebfec17 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:23.366011 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd965e25-c669-41d5-abc8-d4bf8f2596a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:23.366792 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd965e25-c669-41d5-abc8-d4bf8f2596a5 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:23.367080 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd965e25-c669-41d5-abc8-d4bf8f2596a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:23.367365 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd965e25-c669-41d5-abc8-d4bf8f2596a5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:23.368158 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd965e25-c669-41d5-abc8-d4bf8f2596a5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:23.369621 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd965e25-c669-41d5-abc8-d4bf8f2596a5 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:23.401777 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 524/1062] 10.222.0.87 () {60 vars in 1144 bytes} [Tue Jun 21 18:14:23 2022] POST /identity/v3/endpoints => generated 327 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:14:23.499577 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e01a29-075a-436a-b4ca-098afee1c699 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:23.568741 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8642c-08cc-409d-99c5-1570186c2a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:23.569691 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8642c-08cc-409d-99c5-1570186c2a5a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:23.570131 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8642c-08cc-409d-99c5-1570186c2a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:23.570556 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8642c-08cc-409d-99c5-1570186c2a5a None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:23.571573 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d8642c-08cc-409d-99c5-1570186c2a5a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:23.572020 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d8642c-08cc-409d-99c5-1570186c2a5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:23.574064 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d8642c-08cc-409d-99c5-1570186c2a5a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:23.586941 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 539/1063] 10.222.0.87 () {56 vars in 1115 bytes} [Tue Jun 21 18:14:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:14:23.594288 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd965e25-c669-41d5-abc8-d4bf8f2596a5 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:23.806535 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd965e25-c669-41d5-abc8-d4bf8f2596a5 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:23.874211 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:23.899277 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2ce36d-8bfb-4b06-bcfb-91bb59154c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:23.900736 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ce36d-8bfb-4b06-bcfb-91bb59154c12 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:23.901236 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ce36d-8bfb-4b06-bcfb-91bb59154c12 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:23.901827 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ce36d-8bfb-4b06-bcfb-91bb59154c12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:23.925667 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2ce36d-8bfb-4b06-bcfb-91bb59154c12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:23.928639 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2ce36d-8bfb-4b06-bcfb-91bb59154c12 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:23.965695 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-5a2ce36d-8bfb-4b06-bcfb-91bb59154c12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:14:23.965695 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:14:23.965695 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:14:23.965695 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:14:23.965695 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:14:23.965695 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:14:23.965695 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:14:23.965695 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:14:23.965695 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:14:23.965695 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:14:23.965695 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:14:23.965695 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:14:23.965695 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:14:23.965695 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:14:23.965695 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:14:23.967853 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:14:23.967853 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:14:23.967853 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:14:23.967853 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:14:23.967853 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:14:23.967853 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:14:23.967853 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:14:23.967853 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:14:23.967853 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:14:23.967853 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:14:23.967853 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:14:23.967853 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:14:23.967853 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:14:23.967853 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:14:23.967853 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:14:23.967853 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:14:23.967853 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:14:23.977854 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:14:23.977854 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:14:23.977854 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:14:23.977854 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:14:23.977854 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 525/1064] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:14:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:14:23.980923 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8642c-08cc-409d-99c5-1570186c2a5a None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:24.084397 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:24.200995 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8642c-08cc-409d-99c5-1570186c2a5a None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:24.252740 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f50b88f-5611-47df-8385-5421741dd4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:24.254212 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50b88f-5611-47df-8385-5421741dd4bf None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:24.254887 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50b88f-5611-47df-8385-5421741dd4bf None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:24.255387 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50b88f-5611-47df-8385-5421741dd4bf None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:24.256322 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f50b88f-5611-47df-8385-5421741dd4bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:24.256785 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f50b88f-5611-47df-8385-5421741dd4bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:24.258619 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f50b88f-5611-47df-8385-5421741dd4bf None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:24.289045 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 540/1065] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:14:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:14:24.298693 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2ce36d-8bfb-4b06-bcfb-91bb59154c12 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:24.500805 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9c874-dadf-4cbb-8f77-0c77168a0463 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:24.501284 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9c874-dadf-4cbb-8f77-0c77168a0463 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:24.501610 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9c874-dadf-4cbb-8f77-0c77168a0463 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:24.504343 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 541/1066] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:14:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:14:24.513779 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4b0fa-ea63-45fa-b5fe-08fc84638601 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:24.514274 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4b0fa-ea63-45fa-b5fe-08fc84638601 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:24.514519 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4b0fa-ea63-45fa-b5fe-08fc84638601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:24.539280 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-3ff4b0fa-ea63-45fa-b5fe-08fc84638601 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:14:24.540950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3ff4b0fa-ea63-45fa-b5fe-08fc84638601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:14:24.544316 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2ce36d-8bfb-4b06-bcfb-91bb59154c12 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:24.631590 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b785ebfe-a839-4537-a38e-be6272c42060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:24.633111 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b785ebfe-a839-4537-a38e-be6272c42060 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:24.633688 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b785ebfe-a839-4537-a38e-be6272c42060 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:24.634173 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b785ebfe-a839-4537-a38e-be6272c42060 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:24.651823 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b785ebfe-a839-4537-a38e-be6272c42060 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:24.654766 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b785ebfe-a839-4537-a38e-be6272c42060 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:24.685583 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-b785ebfe-a839-4537-a38e-be6272c42060 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:14:24.685583 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:14:24.685583 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:14:24.685583 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:14:24.685583 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:14:24.685583 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:14:24.685583 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:14:24.685583 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:14:24.685583 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:14:24.685583 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:14:24.685583 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:14:24.685583 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:14:24.685583 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:14:24.685583 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:14:24.685583 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:14:24.685583 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:14:24.687498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:14:24.687498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:14:24.687498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:14:24.687498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:14:24.687498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:14:24.687498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:14:24.687498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:14:24.687498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:14:24.687498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:14:24.687498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:14:24.687498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:14:24.687498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:14:24.687498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:14:24.687498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:14:24.687498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:14:24.687498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:14:24.687498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:14:24.687498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:14:24.689392 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:14:24.689392 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:14:24.689392 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:14:24.689392 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:14:24.689392 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:14:24.689392 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:14:24.689392 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:14:24.689392 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:14:24.689392 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:14:24.689392 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:14:24.690472 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 542/1067] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:14:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:14:24.694196 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 526/1068] 10.222.0.87 () {56 vars in 1093 bytes} [Tue Jun 21 18:14:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 396 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:14:24.697830 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff4b0fa-ea63-45fa-b5fe-08fc84638601 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:24.726785 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b785ebfe-a839-4537-a38e-be6272c42060 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:24.768871 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:24.805330 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:24.926195 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff4b0fa-ea63-45fa-b5fe-08fc84638601 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:24.957462 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b785ebfe-a839-4537-a38e-be6272c42060 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:25.004350 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff6bd89-ba8f-434e-9e04-2f9d6902b7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:25.005581 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6bd89-ba8f-434e-9e04-2f9d6902b7b2 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:25.006127 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6bd89-ba8f-434e-9e04-2f9d6902b7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:25.006626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6bd89-ba8f-434e-9e04-2f9d6902b7b2 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:25.008302 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff6bd89-ba8f-434e-9e04-2f9d6902b7b2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:25.008866 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff6bd89-ba8f-434e-9e04-2f9d6902b7b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:25.011833 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff6bd89-ba8f-434e-9e04-2f9d6902b7b2 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:25.035391 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f00231-96e6-41a1-9bd6-6779e59440a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:25.038422 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f00231-96e6-41a1-9bd6-6779e59440a3 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:25.038422 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f00231-96e6-41a1-9bd6-6779e59440a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:25.038422 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f00231-96e6-41a1-9bd6-6779e59440a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:25.049469 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 543/1069] 10.222.0.87 () {56 vars in 1133 bytes} [Tue Jun 21 18:14:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:14:25.058801 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff6bd89-ba8f-434e-9e04-2f9d6902b7b2 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:25.061763 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f00231-96e6-41a1-9bd6-6779e59440a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:14:25.079907 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-86f00231-96e6-41a1-9bd6-6779e59440a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:14:25.079907 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:14:25.079907 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:14:25.079907 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:14:25.079907 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:14:25.079907 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:14:25.079907 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:14:25.079907 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:14:25.079907 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:14:25.079907 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:14:25.079907 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:14:25.079907 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:14:25.079907 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:14:25.079907 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:14:25.079907 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:14:25.079907 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:14:25.081861 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:14:25.081861 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:14:25.081861 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:14:25.081861 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:14:25.081861 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:14:25.081861 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:14:25.081861 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:14:25.081861 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:14:25.081861 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:14:25.081861 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:14:25.081861 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:14:25.081861 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:14:25.081861 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:14:25.081861 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:14:25.081861 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:14:25.081861 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:14:25.083570 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:14:25.083570 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:14:25.083570 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:14:25.083570 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:14:25.083570 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:14:25.083570 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:14:25.083570 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:14:25.083570 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:14:25.083570 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:14:25.083570 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:14:25.083570 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:14:25.083570 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:14:25.083570 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 527/1070] 10.222.0.87 () {56 vars in 1096 bytes} [Tue Jun 21 18:14:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:14:25.093685 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f00231-96e6-41a1-9bd6-6779e59440a3 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:25.221122 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:25.246013 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff6bd89-ba8f-434e-9e04-2f9d6902b7b2 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:25.279705 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:25.291799 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f00231-96e6-41a1-9bd6-6779e59440a3 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:25.320196 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e203ef1-2581-409e-a95b-872d11d968de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:25.321735 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e203ef1-2581-409e-a95b-872d11d968de None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:25.322253 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e203ef1-2581-409e-a95b-872d11d968de None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:25.322758 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e203ef1-2581-409e-a95b-872d11d968de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:25.346249 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e203ef1-2581-409e-a95b-872d11d968de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:25.349193 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e203ef1-2581-409e-a95b-872d11d968de None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:25.368009 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458d2a57-e08a-4dd3-8712-631a5a74828e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:25.369352 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d2a57-e08a-4dd3-8712-631a5a74828e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:25.369930 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d2a57-e08a-4dd3-8712-631a5a74828e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:25.370463 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d2a57-e08a-4dd3-8712-631a5a74828e None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:25.372143 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458d2a57-e08a-4dd3-8712-631a5a74828e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:25.372769 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458d2a57-e08a-4dd3-8712-631a5a74828e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:25.376650 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458d2a57-e08a-4dd3-8712-631a5a74828e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:25.389059 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-4e203ef1-2581-409e-a95b-872d11d968de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:14:25.389059 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:14:25.389059 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:14:25.389059 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:14:25.389059 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:14:25.389059 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:14:25.389059 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:14:25.389059 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:14:25.389059 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:14:25.389059 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:14:25.389059 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:14:25.389059 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:14:25.389059 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:14:25.389059 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:14:25.389059 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:14:25.390982 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:14:25.390982 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:14:25.390982 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:14:25.390982 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:14:25.390982 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:14:25.390982 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:14:25.390982 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:14:25.390982 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:14:25.390982 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:14:25.390982 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:14:25.390982 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:14:25.390982 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:14:25.390982 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:14:25.390982 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:14:25.390982 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:14:25.390982 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:14:25.390982 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:14:25.393088 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:14:25.393088 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:14:25.393088 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:14:25.393088 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:14:25.404522 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 528/1071] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:14:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:14:25.405005 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 544/1072] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:14:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:14:25.405251 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458d2a57-e08a-4dd3-8712-631a5a74828e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:25.406329 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e203ef1-2581-409e-a95b-872d11d968de None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:25.593251 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458d2a57-e08a-4dd3-8712-631a5a74828e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:25.630329 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e203ef1-2581-409e-a95b-872d11d968de None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:25.661268 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed059dd-0a48-4675-ad23-61013c1d0507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:25.662524 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed059dd-0a48-4675-ad23-61013c1d0507 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:25.663034 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed059dd-0a48-4675-ad23-61013c1d0507 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:25.663519 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed059dd-0a48-4675-ad23-61013c1d0507 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:25.664958 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed059dd-0a48-4675-ad23-61013c1d0507 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:25.665518 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed059dd-0a48-4675-ad23-61013c1d0507 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:25.668315 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed059dd-0a48-4675-ad23-61013c1d0507 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:25.699343 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 529/1073] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:14:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:14:25.704150 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed059dd-0a48-4675-ad23-61013c1d0507 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:25.710041 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2210cf21-5149-4d76-971d-1d865e15ff18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:25.712168 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210cf21-5149-4d76-971d-1d865e15ff18 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:25.712168 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210cf21-5149-4d76-971d-1d865e15ff18 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:25.713203 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210cf21-5149-4d76-971d-1d865e15ff18 None admin] PATH_INFO: `/v3/users/02e645c4c6e14786b95e1548c19514ec` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:25.715217 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2210cf21-5149-4d76-971d-1d865e15ff18 None admin] RBAC: Authorizing `identity:get_user(user_id=02e645c4c6e14786b95e1548c19514ec)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:25.718672 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2210cf21-5149-4d76-971d-1d865e15ff18 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:25.728428 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 545/1074] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:14:25 2022] GET /identity/v3/users/02e645c4c6e14786b95e1548c19514ec => generated 285 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:14:25.730126 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2210cf21-5149-4d76-971d-1d865e15ff18 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:25.758161 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:25.759185 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:25.940478 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed059dd-0a48-4675-ad23-61013c1d0507 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:25.957162 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2210cf21-5149-4d76-971d-1d865e15ff18 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:26.028577 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3735ab-3106-470d-bed5-90dd979639aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:26.028872 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c7b74-7323-49ea-aa00-b7709faf9d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:26.030124 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c7b74-7323-49ea-aa00-b7709faf9d74 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:26.030519 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c7b74-7323-49ea-aa00-b7709faf9d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:26.030933 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c7b74-7323-49ea-aa00-b7709faf9d74 None admin] PATH_INFO: `/v3/projects/bd9bd2473fcd4546b4200678e8df9d66` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:26.031755 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3735ab-3106-470d-bed5-90dd979639aa None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:26.032586 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3735ab-3106-470d-bed5-90dd979639aa None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:26.033259 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3735ab-3106-470d-bed5-90dd979639aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:26.033406 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832c7b74-7323-49ea-aa00-b7709faf9d74 None admin] RBAC: Authorizing `identity:get_project(project_id=bd9bd2473fcd4546b4200678e8df9d66)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:26.036093 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832c7b74-7323-49ea-aa00-b7709faf9d74 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:26.046975 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 546/1075] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:14:25 2022] GET /identity/v3/projects/bd9bd2473fcd4546b4200678e8df9d66 => generated 311 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:14:26.050099 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c7b74-7323-49ea-aa00-b7709faf9d74 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:26.050550 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3735ab-3106-470d-bed5-90dd979639aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:26.053009 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3735ab-3106-470d-bed5-90dd979639aa None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:26.076613 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-7a3735ab-3106-470d-bed5-90dd979639aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:14:26.076613 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:14:26.076613 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:14:26.076613 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:14:26.076613 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:14:26.076613 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:14:26.076613 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:14:26.076613 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:14:26.076613 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:14:26.076613 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:14:26.076613 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:14:26.076613 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:14:26.076613 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:14:26.076613 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:14:26.076613 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:14:26.078931 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:14:26.078931 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:14:26.078931 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:14:26.078931 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:14:26.078931 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:14:26.078931 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:14:26.078931 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:14:26.078931 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:14:26.078931 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:14:26.078931 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:14:26.078931 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:14:26.078931 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:14:26.078931 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:14:26.078931 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:14:26.078931 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:14:26.078931 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:14:26.078931 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:14:26.086761 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:14:26.086761 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:14:26.086761 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:14:26.086761 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:14:26.086761 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:14:26.086761 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:14:26.086761 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:14:26.086761 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:14:26.086761 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 530/1076] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:14:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:14:26.088253 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3735ab-3106-470d-bed5-90dd979639aa None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:26.233167 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:26.262005 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:26.276372 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c7b74-7323-49ea-aa00-b7709faf9d74 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:26.298915 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3735ab-3106-470d-bed5-90dd979639aa None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:26.349003 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da72335d-6f91-4694-a492-dc5627985fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:26.350687 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da72335d-6f91-4694-a492-dc5627985fcb None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:26.351178 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da72335d-6f91-4694-a492-dc5627985fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:26.351658 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da72335d-6f91-4694-a492-dc5627985fcb None admin] PATH_INFO: `/v3/projects/bd9bd2473fcd4546b4200678e8df9d66/users/02e645c4c6e14786b95e1548c19514ec/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:26.354715 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe23b1-9a96-4875-bc7c-83231e5003df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:26.355107 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da72335d-6f91-4694-a492-dc5627985fcb None admin] RBAC: Authorizing `identity:create_grant(project_id=bd9bd2473fcd4546b4200678e8df9d66, user_id=02e645c4c6e14786b95e1548c19514ec, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:26.355874 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe23b1-9a96-4875-bc7c-83231e5003df None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:26.356367 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe23b1-9a96-4875-bc7c-83231e5003df None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:26.356853 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe23b1-9a96-4875-bc7c-83231e5003df None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:26.357402 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da72335d-6f91-4694-a492-dc5627985fcb None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:26.358422 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe23b1-9a96-4875-bc7c-83231e5003df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:26.359003 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe23b1-9a96-4875-bc7c-83231e5003df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:26.362940 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe23b1-9a96-4875-bc7c-83231e5003df None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:26.381829 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 547/1077] 10.222.0.87 () {58 vars in 1439 bytes} [Tue Jun 21 18:14:26 2022] PUT /identity/v3/projects/bd9bd2473fcd4546b4200678e8df9d66/users/02e645c4c6e14786b95e1548c19514ec/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:14:26.384326 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25373d2-24d0-4c20-95e0-908afe68830f None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:26.384833 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25373d2-24d0-4c20-95e0-908afe68830f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:26.385326 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25373d2-24d0-4c20-95e0-908afe68830f None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:26.388319 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 548/1078] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:14:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:14:26.397843 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 531/1079] 10.222.0.87 () {56 vars in 1148 bytes} [Tue Jun 21 18:14:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:14:26.401612 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e2b63-2f5f-4b22-ba91-060d80717bd1 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:26.402403 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e2b63-2f5f-4b22-ba91-060d80717bd1 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:26.403019 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e2b63-2f5f-4b22-ba91-060d80717bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:26.411192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe23b1-9a96-4875-bc7c-83231e5003df None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:26.433937 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-7b9e2b63-2f5f-4b22-ba91-060d80717bd1 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:14:26.436132 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7b9e2b63-2f5f-4b22-ba91-060d80717bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:14:26.582305 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 549/1080] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:14:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:14:26.611848 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9e2b63-2f5f-4b22-ba91-060d80717bd1 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:26.634956 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe23b1-9a96-4875-bc7c-83231e5003df None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:26.724010 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdec4130-0f36-4b69-ad0f-5db2fb536537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:26.724010 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec4130-0f36-4b69-ad0f-5db2fb536537 None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:26.726693 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec4130-0f36-4b69-ad0f-5db2fb536537 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:26.726693 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec4130-0f36-4b69-ad0f-5db2fb536537 None admin] PATH_INFO: `/v3/projects/5f30c3ddb5e44e9299ff228fe94c27fb/users/a1b3fa67905c4027b3656ac36e2b76c9/roles/9fa867d83f3c406aabf51ddd8c7ed2aa` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:26.730707 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdec4130-0f36-4b69-ad0f-5db2fb536537 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f30c3ddb5e44e9299ff228fe94c27fb, user_id=a1b3fa67905c4027b3656ac36e2b76c9, role_id=9fa867d83f3c406aabf51ddd8c7ed2aa)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:26.731517 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdec4130-0f36-4b69-ad0f-5db2fb536537 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:26.745343 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:26.745893 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:26.760432 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 532/1081] 10.222.0.87 () {58 vars in 1439 bytes} [Tue Jun 21 18:14:26 2022] PUT /identity/v3/projects/5f30c3ddb5e44e9299ff228fe94c27fb/users/a1b3fa67905c4027b3656ac36e2b76c9/roles/9fa867d83f3c406aabf51ddd8c7ed2aa => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:14:26.827676 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9e2b63-2f5f-4b22-ba91-060d80717bd1 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:26.907467 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfb9484-77f7-422a-bb15-d87cb16914a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:26.908535 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfb9484-77f7-422a-bb15-d87cb16914a4 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:26.909014 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfb9484-77f7-422a-bb15-d87cb16914a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:26.909496 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfb9484-77f7-422a-bb15-d87cb16914a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:27.092031 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfb9484-77f7-422a-bb15-d87cb16914a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:27.095711 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfb9484-77f7-422a-bb15-d87cb16914a4 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:27.256946 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:27.335233 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 550/1082] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:14:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 725 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 21 18:14:27.346833 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdec4130-0f36-4b69-ad0f-5db2fb536537 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:27.587468 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdec4130-0f36-4b69-ad0f-5db2fb536537 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:27.660468 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78fdee5-56dd-4224-9323-50de80321fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:27.661508 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78fdee5-56dd-4224-9323-50de80321fc7 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:27.661831 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78fdee5-56dd-4224-9323-50de80321fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:27.662112 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78fdee5-56dd-4224-9323-50de80321fc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:27.679915 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78fdee5-56dd-4224-9323-50de80321fc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:27.681812 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78fdee5-56dd-4224-9323-50de80321fc7 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:27.710412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-f78fdee5-56dd-4224-9323-50de80321fc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:14:27.710412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:14:27.710412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:14:27.710412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:14:27.710412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:14:27.710412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:14:27.710412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:14:27.710412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:14:27.710412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:14:27.710412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:14:27.710412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:14:27.710412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:14:27.710412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:14:27.710412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:14:27.710412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:14:27.712180 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:14:27.712180 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:14:27.712180 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:14:27.712180 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:14:27.712180 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:14:27.712180 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:14:27.712180 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:14:27.712180 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:14:27.712180 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:14:27.712180 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:14:27.712180 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:14:27.712180 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:14:27.712180 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:14:27.712180 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:14:27.712180 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:14:27.712180 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:14:27.712180 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:14:27.713580 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:14:27.713580 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:14:27.713580 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:14:27.713580 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:14:27.713580 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 533/1083] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:14:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:14:27.724651 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfb9484-77f7-422a-bb15-d87cb16914a4 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:27.907343 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfb9484-77f7-422a-bb15-d87cb16914a4 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:27.966736 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0941c47-7f99-4f46-b03d-e7e4aea7a625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:27.967866 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0941c47-7f99-4f46-b03d-e7e4aea7a625 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:27.968308 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0941c47-7f99-4f46-b03d-e7e4aea7a625 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:27.968689 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0941c47-7f99-4f46-b03d-e7e4aea7a625 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:27.969664 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0941c47-7f99-4f46-b03d-e7e4aea7a625 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:27.970245 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0941c47-7f99-4f46-b03d-e7e4aea7a625 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:27.972694 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0941c47-7f99-4f46-b03d-e7e4aea7a625 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:27.997549 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 551/1084] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:14:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:14:29.271355 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d637bb-eb49-4016-8f88-0774fad37f81 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:29.272065 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d637bb-eb49-4016-8f88-0774fad37f81 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:29.272065 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d637bb-eb49-4016-8f88-0774fad37f81 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:29.273803 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 534/1085] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:14:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:14:29.281736 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d0a22-3fe3-49fb-89ef-68ee8b9f0685 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:29.285040 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d0a22-3fe3-49fb-89ef-68ee8b9f0685 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:29.285040 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d0a22-3fe3-49fb-89ef-68ee8b9f0685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:29.313579 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-ae0d0a22-3fe3-49fb-89ef-68ee8b9f0685 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:14:29.315739 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ae0d0a22-3fe3-49fb-89ef-68ee8b9f0685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:14:29.364817 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:29.450942 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 552/1086] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:14:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:14:29.497019 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d637bb-eb49-4016-8f88-0774fad37f81 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:29.607907 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:29.722647 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d637bb-eb49-4016-8f88-0774fad37f81 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:29.790406 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e1f79-b153-4b79-93f1-084205bcf5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:29.791405 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e1f79-b153-4b79-93f1-084205bcf5fd None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:29.791599 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e1f79-b153-4b79-93f1-084205bcf5fd None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:29.791823 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e1f79-b153-4b79-93f1-084205bcf5fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:29.808871 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640e1f79-b153-4b79-93f1-084205bcf5fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:14:29.819806 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-640e1f79-b153-4b79-93f1-084205bcf5fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:14:29.819806 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:14:29.819806 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:14:29.819806 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:14:29.819806 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:14:29.819806 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:14:29.819806 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:14:29.819806 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:14:29.819806 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:14:29.819806 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:14:29.819806 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:14:29.819806 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:14:29.819806 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:14:29.819806 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:14:29.819806 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:14:29.819806 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:14:29.822252 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:14:29.822252 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:14:29.822252 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:14:29.822252 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:14:29.822252 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:14:29.822252 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:14:29.822252 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:14:29.822252 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:14:29.822252 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:14:29.822252 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:14:29.822252 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:14:29.822252 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:14:29.822252 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:14:29.822252 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:14:29.822252 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:14:29.822252 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:14:29.824259 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:14:29.824259 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:14:29.824259 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:14:29.824259 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:14:29.824259 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:14:29.824259 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:14:29.824259 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:14:29.824259 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:14:29.824259 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:14:29.824259 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:14:29.824259 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:14:29.824259 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:14:29.824259 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 535/1087] 10.222.0.87 () {56 vars in 1096 bytes} [Tue Jun 21 18:14:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:14:29.832175 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d0a22-3fe3-49fb-89ef-68ee8b9f0685 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:30.067439 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d0a22-3fe3-49fb-89ef-68ee8b9f0685 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:30.153148 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf71572-3e36-4e7c-a2a9-2c446f57d4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:30.154552 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf71572-3e36-4e7c-a2a9-2c446f57d4dc None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:30.155057 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf71572-3e36-4e7c-a2a9-2c446f57d4dc None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:30.155555 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf71572-3e36-4e7c-a2a9-2c446f57d4dc None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:30.157980 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf71572-3e36-4e7c-a2a9-2c446f57d4dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:30.158571 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf71572-3e36-4e7c-a2a9-2c446f57d4dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:30.161167 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf71572-3e36-4e7c-a2a9-2c446f57d4dc None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:30.185508 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 553/1088] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:14:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:14:30.188022 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e1f79-b153-4b79-93f1-084205bcf5fd None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:30.201348 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90431208-d2ef-49ee-8785-c6d832d1bf05 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:30.201748 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90431208-d2ef-49ee-8785-c6d832d1bf05 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:30.202137 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90431208-d2ef-49ee-8785-c6d832d1bf05 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:30.207612 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 554/1089] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:14:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:14:30.211296 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f5191-50b5-4ac6-8b32-0dd9b292a519 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:30.211684 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f5191-50b5-4ac6-8b32-0dd9b292a519 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:30.212073 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f5191-50b5-4ac6-8b32-0dd9b292a519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:30.237978 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-fb1f5191-50b5-4ac6-8b32-0dd9b292a519 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:14:30.239481 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-fb1f5191-50b5-4ac6-8b32-0dd9b292a519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:14:30.267345 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:30.360260 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 555/1090] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:14:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:14:30.396756 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e1f79-b153-4b79-93f1-084205bcf5fd None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:30.405586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f5191-50b5-4ac6-8b32-0dd9b292a519 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:30.416264 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:30.471366 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec20739-dd3d-4814-803a-e000eeb329e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:30.472779 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec20739-dd3d-4814-803a-e000eeb329e7 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:30.472779 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec20739-dd3d-4814-803a-e000eeb329e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:30.473618 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec20739-dd3d-4814-803a-e000eeb329e7 None admin] PATH_INFO: `/v3/users/02e645c4c6e14786b95e1548c19514ec` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:30.476026 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec20739-dd3d-4814-803a-e000eeb329e7 None admin] RBAC: Authorizing `identity:get_user(user_id=02e645c4c6e14786b95e1548c19514ec)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:30.478831 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec20739-dd3d-4814-803a-e000eeb329e7 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:30.482528 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 536/1091] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:14:30 2022] GET /identity/v3/users/02e645c4c6e14786b95e1548c19514ec => generated 285 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:14:30.493041 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec20739-dd3d-4814-803a-e000eeb329e7 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:30.618512 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f5191-50b5-4ac6-8b32-0dd9b292a519 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:30.704964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247e0815-a14a-4a2f-a72b-2c66547ceb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:30.706634 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e0815-a14a-4a2f-a72b-2c66547ceb9d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:30.707126 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e0815-a14a-4a2f-a72b-2c66547ceb9d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:30.707622 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e0815-a14a-4a2f-a72b-2c66547ceb9d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:30.718649 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec20739-dd3d-4814-803a-e000eeb329e7 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:30.729502 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247e0815-a14a-4a2f-a72b-2c66547ceb9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:14:30.747740 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-247e0815-a14a-4a2f-a72b-2c66547ceb9d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 18:14:30.747740 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:14:30.747740 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:14:30.747740 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:14:30.747740 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:14:30.747740 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:14:30.747740 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:14:30.747740 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:14:30.747740 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:14:30.747740 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:14:30.747740 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:14:30.747740 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:14:30.747740 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:14:30.747740 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:14:30.747740 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:14:30.747740 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:14:30.749269 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:14:30.749269 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:14:30.749269 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:14:30.749269 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:14:30.749269 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:14:30.749269 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:14:30.749269 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:14:30.749269 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:14:30.749269 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:14:30.749269 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:14:30.749269 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:14:30.749269 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:14:30.749269 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:14:30.749269 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:14:30.749269 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:14:30.749269 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:14:30.750612 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:14:30.750612 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:14:30.750612 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:14:30.750612 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:14:30.750612 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:14:30.750612 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:14:30.750612 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:14:30.750612 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:14:30.750612 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:14:30.750612 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:14:30.750612 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 18:14:30.750612 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:14:30.750612 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 556/1092] 10.222.0.87 () {56 vars in 1120 bytes} [Tue Jun 21 18:14:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:14:30.759305 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247e0815-a14a-4a2f-a72b-2c66547ceb9d None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:30.773461 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7882e281-c589-4221-b43f-b31c6e5355ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:30.774271 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7882e281-c589-4221-b43f-b31c6e5355ac None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:30.774614 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7882e281-c589-4221-b43f-b31c6e5355ac None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:30.774964 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7882e281-c589-4221-b43f-b31c6e5355ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:30.789458 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:30.863050 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:30.911894 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7882e281-c589-4221-b43f-b31c6e5355ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:30.914550 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7882e281-c589-4221-b43f-b31c6e5355ac None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:30.972867 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247e0815-a14a-4a2f-a72b-2c66547ceb9d None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:31.040285 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4495eb80-fbbe-46e5-8d14-a52331d5c622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:31.041261 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495eb80-fbbe-46e5-8d14-a52331d5c622 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:31.041698 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495eb80-fbbe-46e5-8d14-a52331d5c622 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:31.041964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495eb80-fbbe-46e5-8d14-a52331d5c622 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:31.042770 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4495eb80-fbbe-46e5-8d14-a52331d5c622 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:31.043084 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4495eb80-fbbe-46e5-8d14-a52331d5c622 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:31.044541 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4495eb80-fbbe-46e5-8d14-a52331d5c622 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:31.065470 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 557/1093] 10.222.0.87 () {56 vars in 1115 bytes} [Tue Jun 21 18:14:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:14:31.069217 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4495eb80-fbbe-46e5-8d14-a52331d5c622 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:31.121977 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 537/1094] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:14:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 630 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:14:31.129322 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7882e281-c589-4221-b43f-b31c6e5355ac None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:31.211662 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:31.253964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4495eb80-fbbe-46e5-8d14-a52331d5c622 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:31.258604 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:31.293403 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7882e281-c589-4221-b43f-b31c6e5355ac None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:31.316814 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d731ea-b087-4c64-8ae2-de15f01d370b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:31.318106 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d731ea-b087-4c64-8ae2-de15f01d370b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:31.318757 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d731ea-b087-4c64-8ae2-de15f01d370b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:31.319120 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d731ea-b087-4c64-8ae2-de15f01d370b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:31.342530 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d731ea-b087-4c64-8ae2-de15f01d370b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:31.344867 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d731ea-b087-4c64-8ae2-de15f01d370b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:31.348130 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a4567a-9d60-4aac-a456-bb995b5f311b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:31.349201 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a4567a-9d60-4aac-a456-bb995b5f311b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:31.349482 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a4567a-9d60-4aac-a456-bb995b5f311b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:31.349852 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a4567a-9d60-4aac-a456-bb995b5f311b None admin] PATH_INFO: `/v3/projects/bd9bd2473fcd4546b4200678e8df9d66` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:31.351455 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a4567a-9d60-4aac-a456-bb995b5f311b None admin] RBAC: Authorizing `identity:get_project(project_id=bd9bd2473fcd4546b4200678e8df9d66)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:31.353053 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a4567a-9d60-4aac-a456-bb995b5f311b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:31.355075 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 538/1095] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:14:31 2022] GET /identity/v3/projects/bd9bd2473fcd4546b4200678e8df9d66 => generated 311 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:14:31.363063 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a4567a-9d60-4aac-a456-bb995b5f311b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:31.371611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-a0d731ea-b087-4c64-8ae2-de15f01d370b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:14:31.371611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:14:31.371611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:14:31.371611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:14:31.371611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:14:31.371611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:14:31.371611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:14:31.371611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:14:31.371611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:14:31.371611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:14:31.371611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:14:31.371611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:14:31.371611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:14:31.371611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:14:31.371611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:14:31.373344 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:14:31.373344 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:14:31.373344 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:14:31.373344 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:14:31.373344 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:14:31.373344 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:14:31.373344 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:14:31.373344 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:14:31.373344 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:14:31.373344 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:14:31.373344 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:14:31.373344 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:14:31.373344 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:14:31.373344 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:14:31.373344 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:14:31.373344 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:14:31.373344 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:14:31.375155 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:14:31.375155 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:14:31.375155 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:14:31.375155 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:14:31.375155 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 558/1096] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:14:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:14:31.382685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d731ea-b087-4c64-8ae2-de15f01d370b None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:31.542473 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a4567a-9d60-4aac-a456-bb995b5f311b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:31.545526 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d731ea-b087-4c64-8ae2-de15f01d370b None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:31.594426 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:31.606381 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d89d75-873c-4e3a-bc0e-028d81d0799f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:31.607601 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d89d75-873c-4e3a-bc0e-028d81d0799f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:31.608117 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d89d75-873c-4e3a-bc0e-028d81d0799f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:31.608638 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d89d75-873c-4e3a-bc0e-028d81d0799f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:31.610609 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d89d75-873c-4e3a-bc0e-028d81d0799f None admin] RBAC: Adding query filter params (role.id=98f0943a610f4a9e9f8de54f6e9f717d, scope.project.id=bd9bd2473fcd4546b4200678e8df9d66, user.id=02e645c4c6e14786b95e1548c19514ec) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:31.611164 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d89d75-873c-4e3a-bc0e-028d81d0799f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:31.613910 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d89d75-873c-4e3a-bc0e-028d81d0799f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:31.616237 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55510fa9-45fb-446f-aa46-a6bebc849cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:31.616988 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55510fa9-45fb-446f-aa46-a6bebc849cba None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:31.617278 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55510fa9-45fb-446f-aa46-a6bebc849cba None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:31.617580 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55510fa9-45fb-446f-aa46-a6bebc849cba None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:31.618385 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55510fa9-45fb-446f-aa46-a6bebc849cba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:31.618696 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55510fa9-45fb-446f-aa46-a6bebc849cba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:31.620210 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55510fa9-45fb-446f-aa46-a6bebc849cba None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:31.634089 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 539/1097] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:14:31 2022] GET /identity/v3/role_assignments?user.id=02e645c4c6e14786b95e1548c19514ec&scope.project.id=bd9bd2473fcd4546b4200678e8df9d66&role.id=98f0943a610f4a9e9f8de54f6e9f717d => generated 608 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:14:31.639855 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 559/1098] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:14:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:14:31.648635 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d89d75-873c-4e3a-bc0e-028d81d0799f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:31.670455 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:31.841306 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d89d75-873c-4e3a-bc0e-028d81d0799f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:31.909339 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4770a15-22d1-450b-adfb-96ca5c446518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:31.910431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4770a15-22d1-450b-adfb-96ca5c446518 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:31.910748 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4770a15-22d1-450b-adfb-96ca5c446518 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:31.911027 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4770a15-22d1-450b-adfb-96ca5c446518 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:31.923509 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4770a15-22d1-450b-adfb-96ca5c446518 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:31.925192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4770a15-22d1-450b-adfb-96ca5c446518 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:31.945748 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-b4770a15-22d1-450b-adfb-96ca5c446518 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:14:31.945748 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:14:31.945748 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:14:31.945748 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:14:31.945748 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:14:31.945748 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:14:31.945748 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:14:31.945748 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:14:31.945748 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:14:31.945748 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:14:31.945748 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:14:31.945748 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:14:31.945748 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:14:31.945748 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:14:31.945748 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:14:31.945748 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:14:31.947832 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:14:31.947832 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:14:31.947832 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:14:31.947832 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:14:31.947832 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:14:31.947832 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:14:31.947832 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:14:31.947832 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:14:31.947832 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:14:31.947832 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:14:31.947832 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:14:31.947832 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:14:31.947832 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:14:31.947832 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:14:31.947832 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:14:31.947832 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:14:31.947832 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:14:31.947832 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:14:31.950020 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:14:31.950020 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:14:31.950020 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:14:31.950020 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:14:31.950020 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:14:31.950020 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:14:31.950020 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:14:31.950020 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:14:31.950020 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:14:31.950020 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:14:31.950020 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 540/1099] 10.222.0.87 () {56 vars in 1093 bytes} [Tue Jun 21 18:14:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:14:31.955973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55510fa9-45fb-446f-aa46-a6bebc849cba None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:32.161975 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55510fa9-45fb-446f-aa46-a6bebc849cba None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:32.233028 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e0b8f-4c00-4c84-8b16-d265b3fc1eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:32.234639 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e0b8f-4c00-4c84-8b16-d265b3fc1eeb None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:32.235428 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e0b8f-4c00-4c84-8b16-d265b3fc1eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:32.235819 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e0b8f-4c00-4c84-8b16-d265b3fc1eeb None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:32.237601 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83e0b8f-4c00-4c84-8b16-d265b3fc1eeb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:32.238195 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83e0b8f-4c00-4c84-8b16-d265b3fc1eeb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:32.241131 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83e0b8f-4c00-4c84-8b16-d265b3fc1eeb None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:32.272637 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 560/1100] 10.222.0.87 () {56 vars in 1133 bytes} [Tue Jun 21 18:14:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:14:32.278760 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4770a15-22d1-450b-adfb-96ca5c446518 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:32.398258 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:32.460340 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4770a15-22d1-450b-adfb-96ca5c446518 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:32.527580 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc708bf-3519-4c18-9bfa-3df44c8f161e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:32.528587 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc708bf-3519-4c18-9bfa-3df44c8f161e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:32.528992 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc708bf-3519-4c18-9bfa-3df44c8f161e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:32.529382 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc708bf-3519-4c18-9bfa-3df44c8f161e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:32.680099 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc708bf-3519-4c18-9bfa-3df44c8f161e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:32.682903 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc708bf-3519-4c18-9bfa-3df44c8f161e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:32.797420 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:32.862918 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 541/1101] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:14:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:14:32.869823 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e0b8f-4c00-4c84-8b16-d265b3fc1eeb None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:32.941902 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:33.047749 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e0b8f-4c00-4c84-8b16-d265b3fc1eeb None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:33.117856 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a26078-34c4-4fdb-8e8b-ff78b7f9f476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:33.119320 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a26078-34c4-4fdb-8e8b-ff78b7f9f476 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:33.119848 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a26078-34c4-4fdb-8e8b-ff78b7f9f476 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:33.120342 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a26078-34c4-4fdb-8e8b-ff78b7f9f476 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:33.148276 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a26078-34c4-4fdb-8e8b-ff78b7f9f476 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:33.154038 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a26078-34c4-4fdb-8e8b-ff78b7f9f476 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:33.181596 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-c1a26078-34c4-4fdb-8e8b-ff78b7f9f476 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:14:33.181596 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:14:33.181596 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:14:33.181596 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:14:33.181596 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:14:33.181596 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:14:33.181596 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:14:33.181596 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:14:33.181596 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:14:33.181596 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:14:33.181596 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:14:33.181596 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:14:33.181596 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:14:33.181596 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:14:33.181596 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:14:33.183677 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:14:33.183677 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:14:33.183677 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:14:33.183677 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:14:33.183677 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:14:33.183677 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:14:33.183677 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:14:33.183677 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:14:33.183677 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:14:33.183677 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:14:33.183677 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:14:33.183677 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:14:33.183677 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:14:33.183677 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:14:33.183677 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:14:33.183677 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:14:33.183677 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:14:33.185741 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:14:33.185741 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:14:33.185741 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:14:33.185741 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:14:33.189944 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 561/1102] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:14:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:14:33.192350 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc708bf-3519-4c18-9bfa-3df44c8f161e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:33.368736 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc708bf-3519-4c18-9bfa-3df44c8f161e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:33.437259 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfbaf7f-3350-4956-96f3-88316fafe8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:33.438346 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfbaf7f-3350-4956-96f3-88316fafe8ea None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:33.438757 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfbaf7f-3350-4956-96f3-88316fafe8ea None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:33.439044 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfbaf7f-3350-4956-96f3-88316fafe8ea None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:33.440154 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfbaf7f-3350-4956-96f3-88316fafe8ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:33.440761 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfbaf7f-3350-4956-96f3-88316fafe8ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:33.443109 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfbaf7f-3350-4956-96f3-88316fafe8ea None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:33.458672 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 542/1103] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:14:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:14:33.465925 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a26078-34c4-4fdb-8e8b-ff78b7f9f476 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:33.628985 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:33.636697 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a26078-34c4-4fdb-8e8b-ff78b7f9f476 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:33.703401 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb59ba-8fee-4c71-b33a-6f741f849db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:33.704453 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb59ba-8fee-4c71-b33a-6f741f849db9 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:33.704819 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb59ba-8fee-4c71-b33a-6f741f849db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:33.705170 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb59ba-8fee-4c71-b33a-6f741f849db9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:33.713534 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfb59ba-8fee-4c71-b33a-6f741f849db9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:33.716082 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfb59ba-8fee-4c71-b33a-6f741f849db9 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:33.733159 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-fcfb59ba-8fee-4c71-b33a-6f741f849db9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:14:33.733159 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:14:33.733159 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:14:33.733159 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:14:33.733159 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:14:33.733159 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:14:33.733159 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:14:33.733159 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:14:33.733159 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:14:33.733159 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:14:33.733159 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:14:33.733159 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:14:33.733159 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:14:33.733159 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:14:33.733159 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:14:33.735098 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:14:33.735098 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:14:33.735098 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:14:33.735098 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:14:33.735098 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:14:33.735098 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:14:33.735098 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:14:33.735098 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:14:33.735098 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:14:33.735098 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:14:33.735098 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:14:33.735098 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:14:33.735098 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:14:33.735098 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:14:33.735098 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:14:33.735098 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:14:33.735098 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:14:33.736834 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:14:33.736834 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:14:33.736834 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:14:33.736834 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:14:33.736834 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:14:33.736834 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:14:33.736834 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:14:33.736834 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:14:33.740881 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 562/1104] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:14:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:14:33.742514 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfbaf7f-3350-4956-96f3-88316fafe8ea None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:33.793972 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:33.906927 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfbaf7f-3350-4956-96f3-88316fafe8ea None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:33.956351 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8dfa38-bf79-42a9-8d1a-74b3044bb6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:33.957522 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8dfa38-bf79-42a9-8d1a-74b3044bb6c3 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:33.958065 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8dfa38-bf79-42a9-8d1a-74b3044bb6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:33.958564 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8dfa38-bf79-42a9-8d1a-74b3044bb6c3 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:33.960088 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8dfa38-bf79-42a9-8d1a-74b3044bb6c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:33.960631 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8dfa38-bf79-42a9-8d1a-74b3044bb6c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:33.963634 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8dfa38-bf79-42a9-8d1a-74b3044bb6c3 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:33.987223 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 543/1105] 10.222.0.87 () {56 vars in 1148 bytes} [Tue Jun 21 18:14:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:14:33.994526 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb59ba-8fee-4c71-b33a-6f741f849db9 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:34.201595 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb59ba-8fee-4c71-b33a-6f741f849db9 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:34.251421 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f713e3-c3a9-46c8-b4f2-6515ec47ba44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:34.252331 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f713e3-c3a9-46c8-b4f2-6515ec47ba44 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:34.252770 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f713e3-c3a9-46c8-b4f2-6515ec47ba44 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:34.253174 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f713e3-c3a9-46c8-b4f2-6515ec47ba44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:34.254571 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f713e3-c3a9-46c8-b4f2-6515ec47ba44 None admin] RBAC: Adding query filter params (role.id=9fa867d83f3c406aabf51ddd8c7ed2aa, scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb, user.id=a1b3fa67905c4027b3656ac36e2b76c9) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:34.255046 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f713e3-c3a9-46c8-b4f2-6515ec47ba44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:34.257257 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f713e3-c3a9-46c8-b4f2-6515ec47ba44 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:34.267403 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 563/1106] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:14:33 2022] GET /identity/v3/role_assignments?user.id=a1b3fa67905c4027b3656ac36e2b76c9&scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb&role.id=9fa867d83f3c406aabf51ddd8c7ed2aa => generated 608 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:14:34.535646 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09198678-e83b-42cc-a772-a51bf7defe9c None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:34.536079 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09198678-e83b-42cc-a772-a51bf7defe9c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:34.536357 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09198678-e83b-42cc-a772-a51bf7defe9c None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:34.538322 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 544/1107] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:14:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:14:34.545308 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54da453-1a8f-48fc-9d90-76abefa58f1c None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:34.545734 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54da453-1a8f-48fc-9d90-76abefa58f1c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:34.546198 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54da453-1a8f-48fc-9d90-76abefa58f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:34.579989 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-e54da453-1a8f-48fc-9d90-76abefa58f1c None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:14:34.581272 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e54da453-1a8f-48fc-9d90-76abefa58f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:14:34.601920 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:34.685591 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 564/1108] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:14:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:14:34.712635 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09198678-e83b-42cc-a772-a51bf7defe9c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:34.898886 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:34.922275 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09198678-e83b-42cc-a772-a51bf7defe9c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:34.987101 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d11b3-09a2-47d5-8e0a-92d488e25dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:34.988179 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d11b3-09a2-47d5-8e0a-92d488e25dac None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:34.988555 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d11b3-09a2-47d5-8e0a-92d488e25dac None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:34.988924 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d11b3-09a2-47d5-8e0a-92d488e25dac None admin] PATH_INFO: `/v3/domains/default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:34.990498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2d11b3-09a2-47d5-8e0a-92d488e25dac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:34.992174 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2d11b3-09a2-47d5-8e0a-92d488e25dac None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:34.994945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 545/1109] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:14:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:14:35.005170 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54da453-1a8f-48fc-9d90-76abefa58f1c None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:35.180065 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54da453-1a8f-48fc-9d90-76abefa58f1c None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:35.232536 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d98e11-27b6-45db-80b1-d96cb133ac0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:35.233175 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d98e11-27b6-45db-80b1-d96cb133ac0e None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:35.233465 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d98e11-27b6-45db-80b1-d96cb133ac0e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:35.233823 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d98e11-27b6-45db-80b1-d96cb133ac0e None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:35.234796 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d98e11-27b6-45db-80b1-d96cb133ac0e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:35.236215 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d98e11-27b6-45db-80b1-d96cb133ac0e None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:35.262322 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 565/1110] 10.222.0.87 () {60 vars in 1132 bytes} [Tue Jun 21 18:14:35 2022] POST /identity/v3/users => generated 286 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:14:38.140246 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8485ec-94a7-4441-b94f-947fb29c2ac4 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:38.141060 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8485ec-94a7-4441-b94f-947fb29c2ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:38.141060 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8485ec-94a7-4441-b94f-947fb29c2ac4 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:38.142581 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 546/1111] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:14:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:14:38.150318 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f87070-66c8-4f5d-a5b7-aca16d53ff72 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:38.151159 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f87070-66c8-4f5d-a5b7-aca16d53ff72 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:38.151319 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f87070-66c8-4f5d-a5b7-aca16d53ff72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:38.169982 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:38.178870 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-05f87070-66c8-4f5d-a5b7-aca16d53ff72 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:14:38.181327 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-05f87070-66c8-4f5d-a5b7-aca16d53ff72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:14:38.291031 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 566/1112] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:14:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:14:38.321203 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8485ec-94a7-4441-b94f-947fb29c2ac4 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:38.478498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8485ec-94a7-4441-b94f-947fb29c2ac4 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:38.535167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91aa3f6-f85a-41d6-81c5-92734a087164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:38.536183 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91aa3f6-f85a-41d6-81c5-92734a087164 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:38.536562 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91aa3f6-f85a-41d6-81c5-92734a087164 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:38.536892 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91aa3f6-f85a-41d6-81c5-92734a087164 None admin] PATH_INFO: `/v3/roles/46e77bb17fc3465496a7a3a29d210042` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:38.542574 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91aa3f6-f85a-41d6-81c5-92734a087164 None admin] RBAC: Authorizing `identity:get_role(role_id=46e77bb17fc3465496a7a3a29d210042)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:38.544106 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91aa3f6-f85a-41d6-81c5-92734a087164 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:38.545780 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 547/1113] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:14:38 2022] GET /identity/v3/roles/46e77bb17fc3465496a7a3a29d210042 => generated 223 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:14:38.555375 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f87070-66c8-4f5d-a5b7-aca16d53ff72 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:38.704315 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f87070-66c8-4f5d-a5b7-aca16d53ff72 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:38.752951 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a7854-470f-49e9-a4ee-648153b26adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:38.754333 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a7854-470f-49e9-a4ee-648153b26adc None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:38.754765 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a7854-470f-49e9-a4ee-648153b26adc None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:38.755118 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a7854-470f-49e9-a4ee-648153b26adc None admin] PATH_INFO: `/v3/users/9aedd12dc7ba4a4888c411f7e8bf0433` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:38.758980 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:38.783175 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1a7854-470f-49e9-a4ee-648153b26adc None admin] RBAC: Authorizing `identity:get_user(user_id=9aedd12dc7ba4a4888c411f7e8bf0433)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:38.785513 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1a7854-470f-49e9-a4ee-648153b26adc None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:38.788780 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 567/1114] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:14:38 2022] GET /identity/v3/users/9aedd12dc7ba4a4888c411f7e8bf0433 => generated 286 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:14:38.799097 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91aa3f6-f85a-41d6-81c5-92734a087164 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:38.845676 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:38.946888 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91aa3f6-f85a-41d6-81c5-92734a087164 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:38.990960 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dce2ef-82f7-49a2-951b-d7ed3d03ed05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:38.991777 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dce2ef-82f7-49a2-951b-d7ed3d03ed05 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:38.992228 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dce2ef-82f7-49a2-951b-d7ed3d03ed05 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:38.992525 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dce2ef-82f7-49a2-951b-d7ed3d03ed05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:39.100641 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dce2ef-82f7-49a2-951b-d7ed3d03ed05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:39.102080 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dce2ef-82f7-49a2-951b-d7ed3d03ed05 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:39.145223 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:39.296657 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 548/1115] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:14:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 21 18:14:39.306263 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a7854-470f-49e9-a4ee-648153b26adc None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:39.466906 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a7854-470f-49e9-a4ee-648153b26adc None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:39.535074 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f0905-fd11-45e5-94b7-641a4d639328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:39.536050 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f0905-fd11-45e5-94b7-641a4d639328 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:39.536379 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f0905-fd11-45e5-94b7-641a4d639328 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:39.536704 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f0905-fd11-45e5-94b7-641a4d639328 None admin] PATH_INFO: `/v3/projects/bd9bd2473fcd4546b4200678e8df9d66` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:39.538264 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4f0905-fd11-45e5-94b7-641a4d639328 None admin] RBAC: Authorizing `identity:get_project(project_id=bd9bd2473fcd4546b4200678e8df9d66)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:39.539826 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4f0905-fd11-45e5-94b7-641a4d639328 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:39.541901 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 568/1116] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:14:39 2022] GET /identity/v3/projects/bd9bd2473fcd4546b4200678e8df9d66 => generated 311 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:14:39.550032 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dce2ef-82f7-49a2-951b-d7ed3d03ed05 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:39.744827 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dce2ef-82f7-49a2-951b-d7ed3d03ed05 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:39.791792 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:39.809059 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822361a8-7813-4f0f-a1e5-22457412a40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:39.809919 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822361a8-7813-4f0f-a1e5-22457412a40a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:39.810304 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822361a8-7813-4f0f-a1e5-22457412a40a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:39.810682 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822361a8-7813-4f0f-a1e5-22457412a40a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:39.811688 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822361a8-7813-4f0f-a1e5-22457412a40a None admin] RBAC: Adding query filter params (role.id=46e77bb17fc3465496a7a3a29d210042, scope.project.id=bd9bd2473fcd4546b4200678e8df9d66, user.id=9aedd12dc7ba4a4888c411f7e8bf0433) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:39.812900 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822361a8-7813-4f0f-a1e5-22457412a40a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:39.814813 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822361a8-7813-4f0f-a1e5-22457412a40a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:39.829393 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 549/1117] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:14:39 2022] GET /identity/v3/role_assignments?user.id=9aedd12dc7ba4a4888c411f7e8bf0433&scope.project.id=bd9bd2473fcd4546b4200678e8df9d66&role.id=46e77bb17fc3465496a7a3a29d210042 => generated 259 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:14:42.332110 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34260627-9728-44ae-ae67-edd146534700 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:42.333011 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34260627-9728-44ae-ae67-edd146534700 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:42.333191 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34260627-9728-44ae-ae67-edd146534700 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:42.335800 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 569/1118] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:14:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:14:42.344577 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09672c-406e-414c-9b46-1286d4e72b47 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:42.345426 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09672c-406e-414c-9b46-1286d4e72b47 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:42.345426 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09672c-406e-414c-9b46-1286d4e72b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:42.375146 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-7c09672c-406e-414c-9b46-1286d4e72b47 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:14:42.376480 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7c09672c-406e-414c-9b46-1286d4e72b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:14:42.482274 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 550/1119] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:14:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:14:42.506764 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34260627-9728-44ae-ae67-edd146534700 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:42.592598 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:42.642705 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34260627-9728-44ae-ae67-edd146534700 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:42.696775 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd2ae4-ddbe-451c-8530-81a6e77a944d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:42.697803 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd2ae4-ddbe-451c-8530-81a6e77a944d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:42.699572 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd2ae4-ddbe-451c-8530-81a6e77a944d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:42.699572 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd2ae4-ddbe-451c-8530-81a6e77a944d None admin] PATH_INFO: `/v3/roles/46e77bb17fc3465496a7a3a29d210042` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:42.699912 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afd2ae4-ddbe-451c-8530-81a6e77a944d None admin] RBAC: Authorizing `identity:get_role(role_id=46e77bb17fc3465496a7a3a29d210042)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:42.701254 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afd2ae4-ddbe-451c-8530-81a6e77a944d None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:42.703214 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 570/1120] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:14:42 2022] GET /identity/v3/roles/46e77bb17fc3465496a7a3a29d210042 => generated 223 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:14:42.709921 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c09672c-406e-414c-9b46-1286d4e72b47 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:42.879358 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c09672c-406e-414c-9b46-1286d4e72b47 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:42.919685 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:42.945761 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf7a0f-2d19-4559-ad33-84ef1005c46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:42.946674 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf7a0f-2d19-4559-ad33-84ef1005c46a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:42.946956 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf7a0f-2d19-4559-ad33-84ef1005c46a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:42.947236 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf7a0f-2d19-4559-ad33-84ef1005c46a None admin] PATH_INFO: `/v3/users/9aedd12dc7ba4a4888c411f7e8bf0433` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:42.948717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcf7a0f-2d19-4559-ad33-84ef1005c46a None admin] RBAC: Authorizing `identity:get_user(user_id=9aedd12dc7ba4a4888c411f7e8bf0433)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:42.950879 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcf7a0f-2d19-4559-ad33-84ef1005c46a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:42.952708 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 551/1121] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:14:42 2022] GET /identity/v3/users/9aedd12dc7ba4a4888c411f7e8bf0433 => generated 286 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:14:42.960552 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd2ae4-ddbe-451c-8530-81a6e77a944d None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:43.085553 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd2ae4-ddbe-451c-8530-81a6e77a944d None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:43.129980 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:43.138131 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8df2864-ef84-4981-a086-41634dc994e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:43.139389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df2864-ef84-4981-a086-41634dc994e4 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:43.139855 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df2864-ef84-4981-a086-41634dc994e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:43.140148 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df2864-ef84-4981-a086-41634dc994e4 None admin] PATH_INFO: `/v3/projects/bd9bd2473fcd4546b4200678e8df9d66` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:43.141910 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8df2864-ef84-4981-a086-41634dc994e4 None admin] RBAC: Authorizing `identity:get_project(project_id=bd9bd2473fcd4546b4200678e8df9d66)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:43.143409 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8df2864-ef84-4981-a086-41634dc994e4 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:43.145272 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 571/1122] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:14:42 2022] GET /identity/v3/projects/bd9bd2473fcd4546b4200678e8df9d66 => generated 311 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:14:43.151943 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf7a0f-2d19-4559-ad33-84ef1005c46a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:43.349900 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf7a0f-2d19-4559-ad33-84ef1005c46a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:43.409274 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6317dc-48d9-4a20-bc90-1b2d0257486e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:43.410222 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6317dc-48d9-4a20-bc90-1b2d0257486e None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:43.410589 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6317dc-48d9-4a20-bc90-1b2d0257486e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:43.410945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6317dc-48d9-4a20-bc90-1b2d0257486e None admin] PATH_INFO: `/v3/projects/bd9bd2473fcd4546b4200678e8df9d66/users/9aedd12dc7ba4a4888c411f7e8bf0433/roles/46e77bb17fc3465496a7a3a29d210042` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:43.413599 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6317dc-48d9-4a20-bc90-1b2d0257486e None admin] RBAC: Authorizing `identity:create_grant(project_id=bd9bd2473fcd4546b4200678e8df9d66, user_id=9aedd12dc7ba4a4888c411f7e8bf0433, role_id=46e77bb17fc3465496a7a3a29d210042)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:43.415717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6317dc-48d9-4a20-bc90-1b2d0257486e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:43.447074 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 552/1123] 10.222.0.87 () {58 vars in 1439 bytes} [Tue Jun 21 18:14:43 2022] PUT /identity/v3/projects/bd9bd2473fcd4546b4200678e8df9d66/users/9aedd12dc7ba4a4888c411f7e8bf0433/roles/46e77bb17fc3465496a7a3a29d210042 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:14:45.909554 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6ca1b-75e6-4613-b6db-5bec2bb15c09 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:45.910175 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6ca1b-75e6-4613-b6db-5bec2bb15c09 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:45.910625 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6ca1b-75e6-4613-b6db-5bec2bb15c09 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:45.913171 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 572/1124] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:14:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:14:45.928117 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472095d9-c14c-4acf-82f0-628c28e50c1c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:45.928979 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472095d9-c14c-4acf-82f0-628c28e50c1c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:45.929141 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472095d9-c14c-4acf-82f0-628c28e50c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:45.952842 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-472095d9-c14c-4acf-82f0-628c28e50c1c None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:14:45.954245 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-472095d9-c14c-4acf-82f0-628c28e50c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:14:46.027830 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:46.066889 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 553/1125] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:14:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:14:46.100301 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff6ca1b-75e6-4613-b6db-5bec2bb15c09 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:46.245674 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff6ca1b-75e6-4613-b6db-5bec2bb15c09 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:46.315612 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0fb2ad-8e5c-4040-bf78-785355e234b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:46.317419 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0fb2ad-8e5c-4040-bf78-785355e234b6 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:46.317898 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0fb2ad-8e5c-4040-bf78-785355e234b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:46.318395 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0fb2ad-8e5c-4040-bf78-785355e234b6 None admin] PATH_INFO: `/v3/roles/46e77bb17fc3465496a7a3a29d210042` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:46.321183 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0fb2ad-8e5c-4040-bf78-785355e234b6 None admin] RBAC: Authorizing `identity:get_role(role_id=46e77bb17fc3465496a7a3a29d210042)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:46.323779 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0fb2ad-8e5c-4040-bf78-785355e234b6 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:46.326486 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 573/1126] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:14:46 2022] GET /identity/v3/roles/46e77bb17fc3465496a7a3a29d210042 => generated 223 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:14:46.336343 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472095d9-c14c-4acf-82f0-628c28e50c1c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:46.520952 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472095d9-c14c-4acf-82f0-628c28e50c1c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:46.590882 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c573a07b-a10a-4c0f-abd9-2453422a7a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:46.592422 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573a07b-a10a-4c0f-abd9-2453422a7a5b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:46.592951 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573a07b-a10a-4c0f-abd9-2453422a7a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:46.593451 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573a07b-a10a-4c0f-abd9-2453422a7a5b None admin] PATH_INFO: `/v3/users/9aedd12dc7ba4a4888c411f7e8bf0433` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:46.595871 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c573a07b-a10a-4c0f-abd9-2453422a7a5b None admin] RBAC: Authorizing `identity:get_user(user_id=9aedd12dc7ba4a4888c411f7e8bf0433)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:46.598899 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c573a07b-a10a-4c0f-abd9-2453422a7a5b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:46.602132 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 554/1127] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:14:46 2022] GET /identity/v3/users/9aedd12dc7ba4a4888c411f7e8bf0433 => generated 286 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:14:46.611198 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0fb2ad-8e5c-4040-bf78-785355e234b6 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:46.704668 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:46.757992 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0fb2ad-8e5c-4040-bf78-785355e234b6 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:46.803840 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1b1d9-2a1b-4322-9197-afda935f98b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:46.804511 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1b1d9-2a1b-4322-9197-afda935f98b8 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:46.804791 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1b1d9-2a1b-4322-9197-afda935f98b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:46.805065 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1b1d9-2a1b-4322-9197-afda935f98b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:46.937054 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e1b1d9-2a1b-4322-9197-afda935f98b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:46.939617 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e1b1d9-2a1b-4322-9197-afda935f98b8 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:47.022991 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:47.079915 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 574/1128] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:14:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:14:47.086609 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c573a07b-a10a-4c0f-abd9-2453422a7a5b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:47.153166 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:47.221290 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c573a07b-a10a-4c0f-abd9-2453422a7a5b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:47.274642 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf3157-ede9-4c36-8b17-119cd81bf8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:47.275626 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf3157-ede9-4c36-8b17-119cd81bf8cb None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:47.275956 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf3157-ede9-4c36-8b17-119cd81bf8cb None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:47.276283 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf3157-ede9-4c36-8b17-119cd81bf8cb None admin] PATH_INFO: `/v3/projects/bd9bd2473fcd4546b4200678e8df9d66` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:47.277684 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bf3157-ede9-4c36-8b17-119cd81bf8cb None admin] RBAC: Authorizing `identity:get_project(project_id=bd9bd2473fcd4546b4200678e8df9d66)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:47.279441 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bf3157-ede9-4c36-8b17-119cd81bf8cb None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:47.281375 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 555/1129] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:14:47 2022] GET /identity/v3/projects/bd9bd2473fcd4546b4200678e8df9d66 => generated 311 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:14:47.288636 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1b1d9-2a1b-4322-9197-afda935f98b8 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:47.472010 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1b1d9-2a1b-4322-9197-afda935f98b8 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:47.515642 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4db2ac-ab5d-4464-a682-4cf3156b12f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:47.516387 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4db2ac-ab5d-4464-a682-4cf3156b12f6 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:47.516715 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4db2ac-ab5d-4464-a682-4cf3156b12f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:47.517038 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4db2ac-ab5d-4464-a682-4cf3156b12f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:47.518048 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4db2ac-ab5d-4464-a682-4cf3156b12f6 None admin] RBAC: Adding query filter params (role.id=46e77bb17fc3465496a7a3a29d210042, scope.project.id=bd9bd2473fcd4546b4200678e8df9d66, user.id=9aedd12dc7ba4a4888c411f7e8bf0433) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:47.518521 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4db2ac-ab5d-4464-a682-4cf3156b12f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:47.520135 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4db2ac-ab5d-4464-a682-4cf3156b12f6 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:47.527704 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 575/1130] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:14:47 2022] GET /identity/v3/role_assignments?user.id=9aedd12dc7ba4a4888c411f7e8bf0433&scope.project.id=bd9bd2473fcd4546b4200678e8df9d66&role.id=46e77bb17fc3465496a7a3a29d210042 => generated 608 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:14:50.130270 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6605ef95-fbc4-42cd-b3fd-b877e8d74b97 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:50.131043 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6605ef95-fbc4-42cd-b3fd-b877e8d74b97 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:50.131043 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6605ef95-fbc4-42cd-b3fd-b877e8d74b97 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:50.132343 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 556/1131] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:14:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:14:50.140338 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c5674-2012-4a09-a32b-37fc1e3f6719 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:50.141221 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c5674-2012-4a09-a32b-37fc1e3f6719 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:50.141221 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c5674-2012-4a09-a32b-37fc1e3f6719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:50.159545 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-cb3c5674-2012-4a09-a32b-37fc1e3f6719 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:14:50.160829 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-cb3c5674-2012-4a09-a32b-37fc1e3f6719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:14:50.190517 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:50.241953 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 576/1132] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:14:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:14:50.267341 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6605ef95-fbc4-42cd-b3fd-b877e8d74b97 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:50.418506 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6605ef95-fbc4-42cd-b3fd-b877e8d74b97 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:50.437062 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:50.465754 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a3c6f0-e18d-4072-a596-3bc1b993906d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:50.466839 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3c6f0-e18d-4072-a596-3bc1b993906d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:50.467323 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3c6f0-e18d-4072-a596-3bc1b993906d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:50.467839 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3c6f0-e18d-4072-a596-3bc1b993906d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:50.469290 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a3c6f0-e18d-4072-a596-3bc1b993906d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:50.470826 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a3c6f0-e18d-4072-a596-3bc1b993906d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:50.472739 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 557/1133] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:14:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:14:50.481501 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3c5674-2012-4a09-a32b-37fc1e3f6719 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:50.618170 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3c5674-2012-4a09-a32b-37fc1e3f6719 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:50.665267 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f32990-61ac-45bc-9e84-114995bfdc7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:50.665998 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f32990-61ac-45bc-9e84-114995bfdc7c None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:50.666286 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f32990-61ac-45bc-9e84-114995bfdc7c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:50.666565 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f32990-61ac-45bc-9e84-114995bfdc7c None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:50.667309 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f32990-61ac-45bc-9e84-114995bfdc7c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:50.668868 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f32990-61ac-45bc-9e84-114995bfdc7c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:50.704725 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 577/1134] 10.222.0.87 () {60 vars in 1141 bytes} [Tue Jun 21 18:14:50 2022] POST /identity/v3/projects => generated 311 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:14:53.750353 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d7b55-b778-477e-9450-503399c5f0ad None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:53.750993 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d7b55-b778-477e-9450-503399c5f0ad None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:53.750993 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d7b55-b778-477e-9450-503399c5f0ad None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:53.752334 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 558/1135] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:14:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:14:53.758347 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119084e-0c36-494b-b0b3-d605c1283e3a None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:53.758866 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119084e-0c36-494b-b0b3-d605c1283e3a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:53.758987 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119084e-0c36-494b-b0b3-d605c1283e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:53.778040 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-1119084e-0c36-494b-b0b3-d605c1283e3a None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:14:53.779402 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-1119084e-0c36-494b-b0b3-d605c1283e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:14:53.788923 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:53.900788 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 578/1136] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:14:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:14:53.928047 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463d7b55-b778-477e-9450-503399c5f0ad None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:54.103068 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463d7b55-b778-477e-9450-503399c5f0ad None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:54.148251 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0645da0f-fb30-4f18-bccd-a1aa2723ff2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:54.149225 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645da0f-fb30-4f18-bccd-a1aa2723ff2f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:54.149510 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645da0f-fb30-4f18-bccd-a1aa2723ff2f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:54.149896 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645da0f-fb30-4f18-bccd-a1aa2723ff2f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:54.153854 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0645da0f-fb30-4f18-bccd-a1aa2723ff2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:54.156326 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0645da0f-fb30-4f18-bccd-a1aa2723ff2f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:54.158174 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 559/1137] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:14:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:14:54.164997 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1119084e-0c36-494b-b0b3-d605c1283e3a None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:54.287036 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1119084e-0c36-494b-b0b3-d605c1283e3a None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:54.328938 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01061a55-02f2-47e3-bf17-a962073b1031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:54.329678 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01061a55-02f2-47e3-bf17-a962073b1031 None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:54.329954 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01061a55-02f2-47e3-bf17-a962073b1031 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:54.330243 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01061a55-02f2-47e3-bf17-a962073b1031 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:54.330988 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01061a55-02f2-47e3-bf17-a962073b1031 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:54.332468 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01061a55-02f2-47e3-bf17-a962073b1031 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:54.350615 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:54.358962 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 579/1138] 10.222.0.87 () {60 vars in 1132 bytes} [Tue Jun 21 18:14:54 2022] POST /identity/v3/users => generated 286 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:14:57.076851 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80376277-71b4-4b54-8340-9f0c8a587ce9 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:57.077540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80376277-71b4-4b54-8340-9f0c8a587ce9 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:57.077667 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80376277-71b4-4b54-8340-9f0c8a587ce9 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:57.080295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 560/1139] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:14:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:14:57.091422 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba1c03-c71d-4bbf-a9a2-28c32d4f85d2 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:57.092518 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba1c03-c71d-4bbf-a9a2-28c32d4f85d2 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:57.093022 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba1c03-c71d-4bbf-a9a2-28c32d4f85d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:57.124010 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-69ba1c03-c71d-4bbf-a9a2-28c32d4f85d2 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:14:57.126185 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-69ba1c03-c71d-4bbf-a9a2-28c32d4f85d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:14:57.261274 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 580/1140] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:14:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:14:57.292126 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80376277-71b4-4b54-8340-9f0c8a587ce9 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:57.374954 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:57.455598 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80376277-71b4-4b54-8340-9f0c8a587ce9 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:57.505003 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b09c75-ce01-407c-8bc4-e7b0c42a089f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:57.505867 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b09c75-ce01-407c-8bc4-e7b0c42a089f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:57.506153 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b09c75-ce01-407c-8bc4-e7b0c42a089f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:57.506428 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b09c75-ce01-407c-8bc4-e7b0c42a089f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:57.521772 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b09c75-ce01-407c-8bc4-e7b0c42a089f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:14:57.531268 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-f9b09c75-ce01-407c-8bc4-e7b0c42a089f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:14:57.531268 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:14:57.531268 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:14:57.531268 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:14:57.531268 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:14:57.531268 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:14:57.531268 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:14:57.531268 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:14:57.531268 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:14:57.531268 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:14:57.531268 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:14:57.531268 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:14:57.531268 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:14:57.531268 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:14:57.531268 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:14:57.531268 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:14:57.533394 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:14:57.533394 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:14:57.533394 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:14:57.533394 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:14:57.533394 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:14:57.533394 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:14:57.533394 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:14:57.533394 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:14:57.533394 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:14:57.533394 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:14:57.533394 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:14:57.533394 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:14:57.533394 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:14:57.533394 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:14:57.533394 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:14:57.533394 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:14:57.535411 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:14:57.535411 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:14:57.535411 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:14:57.535411 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:14:57.535411 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:14:57.535411 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:14:57.535411 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:14:57.535411 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:14:57.535411 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:14:57.535411 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:14:57.535411 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:14:57.535411 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:14:57.535411 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 561/1141] 10.222.0.87 () {56 vars in 1096 bytes} [Tue Jun 21 18:14:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:14:57.541176 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ba1c03-c71d-4bbf-a9a2-28c32d4f85d2 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:57.682009 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ba1c03-c71d-4bbf-a9a2-28c32d4f85d2 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:57.715064 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:57.726641 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b926593f-9042-46d1-bec2-11d7b1080dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:57.727430 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926593f-9042-46d1-bec2-11d7b1080dc6 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:57.727760 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926593f-9042-46d1-bec2-11d7b1080dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:57.728080 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926593f-9042-46d1-bec2-11d7b1080dc6 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:57.728906 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b926593f-9042-46d1-bec2-11d7b1080dc6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:57.729260 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b926593f-9042-46d1-bec2-11d7b1080dc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:57.730796 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b926593f-9042-46d1-bec2-11d7b1080dc6 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:57.741491 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 581/1142] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:14:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:14:57.748598 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b09c75-ce01-407c-8bc4-e7b0c42a089f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:57.942974 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b09c75-ce01-407c-8bc4-e7b0c42a089f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:57.963781 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:58.007432 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec794d7-2110-4bbc-a499-e5f773c988e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:58.008981 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec794d7-2110-4bbc-a499-e5f773c988e7 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:58.009476 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec794d7-2110-4bbc-a499-e5f773c988e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:58.010025 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec794d7-2110-4bbc-a499-e5f773c988e7 None admin] PATH_INFO: `/v3/users/8172589ef13a4daca11722bcaf91ebc8` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:58.039331 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec794d7-2110-4bbc-a499-e5f773c988e7 None admin] RBAC: Authorizing `identity:get_user(user_id=8172589ef13a4daca11722bcaf91ebc8)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:58.042158 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec794d7-2110-4bbc-a499-e5f773c988e7 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:58.046794 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 562/1143] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:14:57 2022] GET /identity/v3/users/8172589ef13a4daca11722bcaf91ebc8 => generated 286 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:14:58.057322 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b926593f-9042-46d1-bec2-11d7b1080dc6 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:58.206812 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b926593f-9042-46d1-bec2-11d7b1080dc6 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:58.255713 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e2214-d720-483c-a04b-56679a2004a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:58.256664 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e2214-d720-483c-a04b-56679a2004a9 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:58.256838 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e2214-d720-483c-a04b-56679a2004a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:58.257099 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e2214-d720-483c-a04b-56679a2004a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:58.337251 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:58.355517 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22e2214-d720-483c-a04b-56679a2004a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:58.358370 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22e2214-d720-483c-a04b-56679a2004a9 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:58.513164 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 582/1144] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:14:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:14:58.523858 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec794d7-2110-4bbc-a499-e5f773c988e7 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:58.726020 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec794d7-2110-4bbc-a499-e5f773c988e7 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:58.770881 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4633f31-7bb0-480c-9431-f41932dda98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:58.771728 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4633f31-7bb0-480c-9431-f41932dda98d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:58.772044 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4633f31-7bb0-480c-9431-f41932dda98d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:58.772368 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4633f31-7bb0-480c-9431-f41932dda98d None admin] PATH_INFO: `/v3/projects/e72f8ff7804a49ca81b6630f74674558` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:58.773732 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4633f31-7bb0-480c-9431-f41932dda98d None admin] RBAC: Authorizing `identity:get_project(project_id=e72f8ff7804a49ca81b6630f74674558)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:58.775240 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4633f31-7bb0-480c-9431-f41932dda98d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:58.777087 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 563/1145] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:14:58 2022] GET /identity/v3/projects/e72f8ff7804a49ca81b6630f74674558 => generated 311 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:14:58.784733 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e2214-d720-483c-a04b-56679a2004a9 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:14:58.907508 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:14:58.922883 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e2214-d720-483c-a04b-56679a2004a9 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:14:58.962284 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d6a29-4c38-4abc-8969-0f85212c71b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:14:58.963298 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d6a29-4c38-4abc-8969-0f85212c71b4 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:14:58.964048 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d6a29-4c38-4abc-8969-0f85212c71b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:14:58.964459 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d6a29-4c38-4abc-8969-0f85212c71b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:14:58.965747 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331d6a29-4c38-4abc-8969-0f85212c71b4 None admin] RBAC: Adding query filter params (role.id=98f0943a610f4a9e9f8de54f6e9f717d, scope.project.id=e72f8ff7804a49ca81b6630f74674558, user.id=8172589ef13a4daca11722bcaf91ebc8) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:14:58.966112 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331d6a29-4c38-4abc-8969-0f85212c71b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:14:58.968115 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331d6a29-4c38-4abc-8969-0f85212c71b4 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:14:58.975604 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 583/1146] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:14:58 2022] GET /identity/v3/role_assignments?user.id=8172589ef13a4daca11722bcaf91ebc8&scope.project.id=e72f8ff7804a49ca81b6630f74674558&role.id=98f0943a610f4a9e9f8de54f6e9f717d => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:15:01.403551 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f5fd8-d779-4a93-99f5-eeeea56b673c None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:01.404287 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f5fd8-d779-4a93-99f5-eeeea56b673c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:01.404528 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f5fd8-d779-4a93-99f5-eeeea56b673c None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:01.407237 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 564/1147] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:15:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:15:01.415508 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecd4b5-0114-4351-a99a-3455b791d1d9 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:01.416108 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecd4b5-0114-4351-a99a-3455b791d1d9 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:01.416276 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecd4b5-0114-4351-a99a-3455b791d1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:01.445050 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-c6ecd4b5-0114-4351-a99a-3455b791d1d9 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:15:01.446629 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c6ecd4b5-0114-4351-a99a-3455b791d1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:15:01.564144 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 584/1148] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:15:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:15:01.594480 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711f5fd8-d779-4a93-99f5-eeeea56b673c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:15:01.650896 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:15:01.787423 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711f5fd8-d779-4a93-99f5-eeeea56b673c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:15:01.849852 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15fecea-a8a5-49ba-aeb6-82090ae0ed7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:15:01.851046 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fecea-a8a5-49ba-aeb6-82090ae0ed7c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:01.851471 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fecea-a8a5-49ba-aeb6-82090ae0ed7c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:01.851891 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fecea-a8a5-49ba-aeb6-82090ae0ed7c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:01.866477 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15fecea-a8a5-49ba-aeb6-82090ae0ed7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:15:01.874334 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-f15fecea-a8a5-49ba-aeb6-82090ae0ed7c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:15:01.874334 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:15:01.874334 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:15:01.874334 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:15:01.874334 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:15:01.874334 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:15:01.874334 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:15:01.874334 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:15:01.874334 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:15:01.874334 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:15:01.874334 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:15:01.874334 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:15:01.874334 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:15:01.874334 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:15:01.874334 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:15:01.874334 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:15:01.876502 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:15:01.876502 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:15:01.876502 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:15:01.876502 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:15:01.876502 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:15:01.876502 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:15:01.876502 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:15:01.876502 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:15:01.876502 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:15:01.876502 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:15:01.876502 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:15:01.876502 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:15:01.876502 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:15:01.876502 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:15:01.876502 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:15:01.876502 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:15:01.878491 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:15:01.878491 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:15:01.878491 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:15:01.878491 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:15:01.878491 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:15:01.878491 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:15:01.878491 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:15:01.878491 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:15:01.878491 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:15:01.878491 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:15:01.878491 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:15:01.878491 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:15:01.878491 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 565/1149] 10.222.0.87 () {56 vars in 1096 bytes} [Tue Jun 21 18:15:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:15:01.884158 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ecd4b5-0114-4351-a99a-3455b791d1d9 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:15:02.016826 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:15:02.042546 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ecd4b5-0114-4351-a99a-3455b791d1d9 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:15:02.100147 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b4c7c-b25e-4b2b-89fd-bc0529e98806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:15:02.100836 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b4c7c-b25e-4b2b-89fd-bc0529e98806 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:02.101119 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b4c7c-b25e-4b2b-89fd-bc0529e98806 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:02.101394 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b4c7c-b25e-4b2b-89fd-bc0529e98806 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:02.102189 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770b4c7c-b25e-4b2b-89fd-bc0529e98806 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:15:02.102499 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770b4c7c-b25e-4b2b-89fd-bc0529e98806 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:15:02.103922 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770b4c7c-b25e-4b2b-89fd-bc0529e98806 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:15:02.115055 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 585/1150] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:15:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:15:02.125072 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15fecea-a8a5-49ba-aeb6-82090ae0ed7c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:15:02.297190 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:15:02.309233 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15fecea-a8a5-49ba-aeb6-82090ae0ed7c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:15:02.363092 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c46f1f-24d9-4498-8476-998689797327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:15:02.364075 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c46f1f-24d9-4498-8476-998689797327 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:02.364413 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c46f1f-24d9-4498-8476-998689797327 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:02.364968 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c46f1f-24d9-4498-8476-998689797327 None admin] PATH_INFO: `/v3/users/8172589ef13a4daca11722bcaf91ebc8` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:02.366329 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c46f1f-24d9-4498-8476-998689797327 None admin] RBAC: Authorizing `identity:get_user(user_id=8172589ef13a4daca11722bcaf91ebc8)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:15:02.368001 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c46f1f-24d9-4498-8476-998689797327 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:15:02.369873 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 566/1151] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:15:02 2022] GET /identity/v3/users/8172589ef13a4daca11722bcaf91ebc8 => generated 286 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:15:02.377345 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b4c7c-b25e-4b2b-89fd-bc0529e98806 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:15:02.548774 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b4c7c-b25e-4b2b-89fd-bc0529e98806 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:15:02.607672 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e83ec1-e7d9-4370-b970-2f553a8e4f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:15:02.608798 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e83ec1-e7d9-4370-b970-2f553a8e4f80 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:02.609180 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e83ec1-e7d9-4370-b970-2f553a8e4f80 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:02.609554 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e83ec1-e7d9-4370-b970-2f553a8e4f80 None admin] PATH_INFO: `/v3/projects/e72f8ff7804a49ca81b6630f74674558` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:02.611315 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e83ec1-e7d9-4370-b970-2f553a8e4f80 None admin] RBAC: Authorizing `identity:get_project(project_id=e72f8ff7804a49ca81b6630f74674558)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:15:02.613111 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e83ec1-e7d9-4370-b970-2f553a8e4f80 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:15:02.615163 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 586/1152] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:15:02 2022] GET /identity/v3/projects/e72f8ff7804a49ca81b6630f74674558 => generated 311 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:15:02.624644 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c46f1f-24d9-4498-8476-998689797327 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:15:02.800735 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c46f1f-24d9-4498-8476-998689797327 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:15:02.857873 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2605f5-4dd9-4e50-bf63-41c558d00c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:15:02.859502 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2605f5-4dd9-4e50-bf63-41c558d00c1c None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:02.860011 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2605f5-4dd9-4e50-bf63-41c558d00c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:02.860732 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2605f5-4dd9-4e50-bf63-41c558d00c1c None admin] PATH_INFO: `/v3/projects/e72f8ff7804a49ca81b6630f74674558/users/8172589ef13a4daca11722bcaf91ebc8/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:02.864326 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2605f5-4dd9-4e50-bf63-41c558d00c1c None admin] RBAC: Authorizing `identity:create_grant(project_id=e72f8ff7804a49ca81b6630f74674558, user_id=8172589ef13a4daca11722bcaf91ebc8, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:15:02.867144 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2605f5-4dd9-4e50-bf63-41c558d00c1c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:15:02.891655 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 567/1153] 10.222.0.87 () {58 vars in 1439 bytes} [Tue Jun 21 18:15:02 2022] PUT /identity/v3/projects/e72f8ff7804a49ca81b6630f74674558/users/8172589ef13a4daca11722bcaf91ebc8/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:15:05.561229 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad6995-c8aa-4be0-a6d5-0298a73efdae None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:05.561924 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad6995-c8aa-4be0-a6d5-0298a73efdae None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:05.561924 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad6995-c8aa-4be0-a6d5-0298a73efdae None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:05.564148 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 587/1154] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:15:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:15:05.570107 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b13a1-7da6-4460-a52b-3276472fb9aa None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:05.570920 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b13a1-7da6-4460-a52b-3276472fb9aa None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:05.571101 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b13a1-7da6-4460-a52b-3276472fb9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:05.592016 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-9d0b13a1-7da6-4460-a52b-3276472fb9aa None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:15:05.593776 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9d0b13a1-7da6-4460-a52b-3276472fb9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:15:05.627490 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:15:05.708622 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 568/1155] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:15:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:15:05.745878 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ad6995-c8aa-4be0-a6d5-0298a73efdae None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:15:05.768140 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:15:05.927068 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ad6995-c8aa-4be0-a6d5-0298a73efdae None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:15:05.969291 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24830d50-be60-4945-b596-b920341a2adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:15:05.970550 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24830d50-be60-4945-b596-b920341a2adc None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:05.970920 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24830d50-be60-4945-b596-b920341a2adc None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:05.971247 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24830d50-be60-4945-b596-b920341a2adc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:05.984683 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24830d50-be60-4945-b596-b920341a2adc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:15:05.993328 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-24830d50-be60-4945-b596-b920341a2adc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:15:05.993328 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:15:05.993328 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:15:05.993328 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:15:05.993328 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:15:05.993328 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:15:05.993328 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:15:05.993328 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:15:05.993328 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:15:05.993328 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:15:05.993328 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:15:05.993328 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:15:05.993328 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:15:05.993328 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:15:05.993328 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:15:05.993328 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:15:05.995606 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:15:05.995606 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:15:05.995606 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:15:05.995606 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:15:05.995606 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:15:05.995606 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:15:05.995606 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:15:05.995606 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:15:05.995606 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:15:05.995606 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:15:05.995606 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:15:05.995606 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:15:05.995606 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:15:05.995606 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:15:05.995606 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:15:05.995606 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:15:05.997086 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:15:05.997086 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:15:05.997086 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:15:05.997086 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:15:05.997086 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:15:05.997086 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:15:05.997086 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:15:05.997086 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:15:05.997086 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:15:05.997086 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:15:05.997086 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:15:05.997086 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:15:05.997086 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 588/1156] 10.222.0.87 () {56 vars in 1096 bytes} [Tue Jun 21 18:15:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:15:06.002932 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0b13a1-7da6-4460-a52b-3276472fb9aa None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:15:06.188082 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0b13a1-7da6-4460-a52b-3276472fb9aa None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:15:06.256963 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c702a-b43e-4566-ad50-d9c974410e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:15:06.258115 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c702a-b43e-4566-ad50-d9c974410e97 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:06.258565 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c702a-b43e-4566-ad50-d9c974410e97 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:06.258994 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c702a-b43e-4566-ad50-d9c974410e97 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:06.260087 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4c702a-b43e-4566-ad50-d9c974410e97 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:15:06.260598 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4c702a-b43e-4566-ad50-d9c974410e97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:15:06.262838 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4c702a-b43e-4566-ad50-d9c974410e97 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:15:06.278405 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 569/1157] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:15:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:15:06.285597 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24830d50-be60-4945-b596-b920341a2adc None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:15:06.381800 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:15:06.420923 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24830d50-be60-4945-b596-b920341a2adc None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:15:06.462986 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94be6078-45e5-49f6-8d6c-6e11ea32c07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:15:06.463940 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be6078-45e5-49f6-8d6c-6e11ea32c07a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:06.464348 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be6078-45e5-49f6-8d6c-6e11ea32c07a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:06.464680 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be6078-45e5-49f6-8d6c-6e11ea32c07a None admin] PATH_INFO: `/v3/users/8172589ef13a4daca11722bcaf91ebc8` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:06.466177 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94be6078-45e5-49f6-8d6c-6e11ea32c07a None admin] RBAC: Authorizing `identity:get_user(user_id=8172589ef13a4daca11722bcaf91ebc8)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:15:06.468110 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94be6078-45e5-49f6-8d6c-6e11ea32c07a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:15:06.470122 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 589/1158] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:15:06 2022] GET /identity/v3/users/8172589ef13a4daca11722bcaf91ebc8 => generated 286 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:15:06.477104 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c702a-b43e-4566-ad50-d9c974410e97 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:15:06.510561 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:15:06.612666 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c702a-b43e-4566-ad50-d9c974410e97 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:15:06.655794 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d5e4aa-0f63-499f-8ace-4cd45813f22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:15:06.656512 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d5e4aa-0f63-499f-8ace-4cd45813f22c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:06.656870 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d5e4aa-0f63-499f-8ace-4cd45813f22c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:06.657197 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d5e4aa-0f63-499f-8ace-4cd45813f22c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:06.736521 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d5e4aa-0f63-499f-8ace-4cd45813f22c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:15:06.737941 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d5e4aa-0f63-499f-8ace-4cd45813f22c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:15:06.759094 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:15:06.902712 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 570/1159] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:15:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:15:06.912592 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94be6078-45e5-49f6-8d6c-6e11ea32c07a None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:15:07.121700 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94be6078-45e5-49f6-8d6c-6e11ea32c07a None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:15:07.190556 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1b60fb-82a9-4cba-a328-f271df6984fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:15:07.192150 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1b60fb-82a9-4cba-a328-f271df6984fe None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:07.192635 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1b60fb-82a9-4cba-a328-f271df6984fe None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:07.193116 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1b60fb-82a9-4cba-a328-f271df6984fe None admin] PATH_INFO: `/v3/projects/e72f8ff7804a49ca81b6630f74674558` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:07.195553 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1b60fb-82a9-4cba-a328-f271df6984fe None admin] RBAC: Authorizing `identity:get_project(project_id=e72f8ff7804a49ca81b6630f74674558)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:15:07.198288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1b60fb-82a9-4cba-a328-f271df6984fe None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:15:07.201977 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 590/1160] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:15:06 2022] GET /identity/v3/projects/e72f8ff7804a49ca81b6630f74674558 => generated 311 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:15:07.211855 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d5e4aa-0f63-499f-8ace-4cd45813f22c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:15:07.422661 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d5e4aa-0f63-499f-8ace-4cd45813f22c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:15:07.456107 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:15:07.490766 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ef9b4d-7974-4f6a-ac63-626a27c72ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:15:07.491919 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef9b4d-7974-4f6a-ac63-626a27c72ca4 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:07.492378 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef9b4d-7974-4f6a-ac63-626a27c72ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:07.492731 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef9b4d-7974-4f6a-ac63-626a27c72ca4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:07.494466 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ef9b4d-7974-4f6a-ac63-626a27c72ca4 None admin] RBAC: Adding query filter params (role.id=98f0943a610f4a9e9f8de54f6e9f717d, scope.project.id=e72f8ff7804a49ca81b6630f74674558, user.id=8172589ef13a4daca11722bcaf91ebc8) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:15:07.495038 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ef9b4d-7974-4f6a-ac63-626a27c72ca4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:15:07.497832 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ef9b4d-7974-4f6a-ac63-626a27c72ca4 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:15:07.508981 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 571/1161] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:15:07 2022] GET /identity/v3/role_assignments?user.id=8172589ef13a4daca11722bcaf91ebc8&scope.project.id=e72f8ff7804a49ca81b6630f74674558&role.id=98f0943a610f4a9e9f8de54f6e9f717d => generated 608 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:15:09.865551 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7bfda9-446f-48da-bed7-bff03d1d629c None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:09.866423 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7bfda9-446f-48da-bed7-bff03d1d629c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:09.866423 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7bfda9-446f-48da-bed7-bff03d1d629c None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:09.868163 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 591/1162] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:15:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:15:09.874022 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ceab6-7519-4fd4-b1f3-cd3473cbf678 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:09.874813 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ceab6-7519-4fd4-b1f3-cd3473cbf678 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:09.874813 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ceab6-7519-4fd4-b1f3-cd3473cbf678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:09.908289 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-660ceab6-7519-4fd4-b1f3-cd3473cbf678 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:15:09.910324 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-660ceab6-7519-4fd4-b1f3-cd3473cbf678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:15:10.026793 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 572/1163] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:15:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:15:10.050695 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7bfda9-446f-48da-bed7-bff03d1d629c None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:15:10.065250 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:15:10.203578 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7bfda9-446f-48da-bed7-bff03d1d629c None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:15:10.258189 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b332868-a82d-41fa-bf27-8f2bba0cf981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:15:10.259027 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b332868-a82d-41fa-bf27-8f2bba0cf981 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:10.259389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b332868-a82d-41fa-bf27-8f2bba0cf981 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:10.259685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b332868-a82d-41fa-bf27-8f2bba0cf981 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:10.357476 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b332868-a82d-41fa-bf27-8f2bba0cf981 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:15:10.359190 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b332868-a82d-41fa-bf27-8f2bba0cf981 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:15:10.373096 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:15:10.496739 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 592/1164] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:15:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:15:10.504916 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ceab6-7519-4fd4-b1f3-cd3473cbf678 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:15:10.667455 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ceab6-7519-4fd4-b1f3-cd3473cbf678 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:15:10.685271 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:15:10.715043 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39926eb2-9f61-45b6-aa15-23a6df31acd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:15:10.716118 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39926eb2-9f61-45b6-aa15-23a6df31acd2 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:10.716508 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39926eb2-9f61-45b6-aa15-23a6df31acd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:10.716901 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39926eb2-9f61-45b6-aa15-23a6df31acd2 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:10.727479 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39926eb2-9f61-45b6-aa15-23a6df31acd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:15:10.730021 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39926eb2-9f61-45b6-aa15-23a6df31acd2 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:15:10.751610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-39926eb2-9f61-45b6-aa15-23a6df31acd2 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 21 18:15:10.751610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:15:10.751610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 21 18:15:10.751610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 21 18:15:10.751610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:15:10.751610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:15:10.751610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:15:10.751610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:15:10.751610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 21 18:15:10.751610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:15:10.751610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:15:10.751610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:15:10.751610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:15:10.751610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:15:10.751610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:15:10.751610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:15:10.751610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:15:10.754408 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:15:10.754408 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:15:10.754408 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:15:10.754408 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:15:10.754408 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:15:10.754408 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:15:10.754408 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:15:10.754408 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:15:10.754408 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:15:10.754408 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:15:10.754408 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:15:10.754408 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:15:10.754408 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:15:10.754408 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:15:10.754408 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:15:10.754408 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:15:10.755692 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:15:10.755692 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:15:10.755692 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:15:10.755692 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:15:10.755692 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:15:10.755692 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:15:10.755692 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:15:10.755692 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:15:10.755692 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:15:10.755692 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:15:10.755692 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 21 18:15:10.755692 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:15:10.755692 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 21 18:15:10.755692 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:15:10.755692 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 573/1165] 10.222.0.87 () {56 vars in 1117 bytes} [Tue Jun 21 18:15:10 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:15:10.759957 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b332868-a82d-41fa-bf27-8f2bba0cf981 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:15:10.953390 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b332868-a82d-41fa-bf27-8f2bba0cf981 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:15:10.968360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:15:11.009277 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87569a1-5d53-4bf1-a0a8-0377f4584270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:15:11.009964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87569a1-5d53-4bf1-a0a8-0377f4584270 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:11.010290 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87569a1-5d53-4bf1-a0a8-0377f4584270 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:11.010609 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87569a1-5d53-4bf1-a0a8-0377f4584270 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:11.011430 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87569a1-5d53-4bf1-a0a8-0377f4584270 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:15:11.011770 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87569a1-5d53-4bf1-a0a8-0377f4584270 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:15:11.013238 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87569a1-5d53-4bf1-a0a8-0377f4584270 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:15:11.022227 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 593/1166] 10.222.0.87 () {56 vars in 1115 bytes} [Tue Jun 21 18:15:10 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:15:13.827708 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a622e-54b1-4ae4-b048-a1bc75358f0b None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:13.828439 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a622e-54b1-4ae4-b048-a1bc75358f0b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:13.828781 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a622e-54b1-4ae4-b048-a1bc75358f0b None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:13.831468 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 574/1167] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:15:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:15:13.842622 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45be12c1-b483-4674-8e6a-b91cce5998d5 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:13.843299 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45be12c1-b483-4674-8e6a-b91cce5998d5 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:13.843941 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45be12c1-b483-4674-8e6a-b91cce5998d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:13.875139 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-45be12c1-b483-4674-8e6a-b91cce5998d5 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:15:13.877335 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-45be12c1-b483-4674-8e6a-b91cce5998d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:15:14.012749 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 594/1168] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:15:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:15:14.047188 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3a622e-54b1-4ae4-b048-a1bc75358f0b None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:15:14.240276 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3a622e-54b1-4ae4-b048-a1bc75358f0b None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:15:14.325158 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1868c376-6a7f-4f24-9255-7bd6b2fe5e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:15:14.326242 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868c376-6a7f-4f24-9255-7bd6b2fe5e02 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:14.326702 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868c376-6a7f-4f24-9255-7bd6b2fe5e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:14.327152 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868c376-6a7f-4f24-9255-7bd6b2fe5e02 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:14.328127 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1868c376-6a7f-4f24-9255-7bd6b2fe5e02 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:15:14.330191 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1868c376-6a7f-4f24-9255-7bd6b2fe5e02 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:15:14.355487 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 575/1169] 10.222.0.87 () {60 vars in 1138 bytes} [Tue Jun 21 18:15:14 2022] POST /identity/v3/domains => generated 267 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:15:17.814292 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc517ea-d57c-4954-84a1-2195465fafda None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:17.815174 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc517ea-d57c-4954-84a1-2195465fafda None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:17.815350 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc517ea-d57c-4954-84a1-2195465fafda None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:17.817909 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 595/1170] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:15:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:15:17.826718 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43384b3-21fa-4038-8ae3-4ef5fcc689e6 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:17.827584 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43384b3-21fa-4038-8ae3-4ef5fcc689e6 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:17.827882 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43384b3-21fa-4038-8ae3-4ef5fcc689e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:17.859630 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-c43384b3-21fa-4038-8ae3-4ef5fcc689e6 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:15:17.862278 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c43384b3-21fa-4038-8ae3-4ef5fcc689e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:15:17.911704 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:15:18.000254 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 576/1171] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:15:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:15:18.033946 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc517ea-d57c-4954-84a1-2195465fafda None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:15:18.205940 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc517ea-d57c-4954-84a1-2195465fafda None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:15:18.211423 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:15:18.274749 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a859a3-0c18-4933-8fa7-0c13c21e208a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:15:18.276302 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a859a3-0c18-4933-8fa7-0c13c21e208a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:18.276805 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a859a3-0c18-4933-8fa7-0c13c21e208a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:18.277307 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a859a3-0c18-4933-8fa7-0c13c21e208a None admin] PATH_INFO: `/v3/domains/f06c4fe3b22e4023919df0dff134c6dc` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:18.299120 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a859a3-0c18-4933-8fa7-0c13c21e208a None admin] RBAC: Authorizing `identity:get_domain(domain_id=f06c4fe3b22e4023919df0dff134c6dc)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:15:18.302091 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a859a3-0c18-4933-8fa7-0c13c21e208a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:15:18.305696 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 596/1172] 10.222.0.87 () {56 vars in 1183 bytes} [Tue Jun 21 18:15:18 2022] GET /identity/v3/domains/f06c4fe3b22e4023919df0dff134c6dc => generated 267 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:15:18.312392 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43384b3-21fa-4038-8ae3-4ef5fcc689e6 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:15:18.488910 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43384b3-21fa-4038-8ae3-4ef5fcc689e6 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:15:18.537864 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbc8a2a-4d44-4486-8832-cb7cee546fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:15:18.538554 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbc8a2a-4d44-4486-8832-cb7cee546fb4 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:18.538839 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbc8a2a-4d44-4486-8832-cb7cee546fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:18.539118 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbc8a2a-4d44-4486-8832-cb7cee546fb4 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:18.540036 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbc8a2a-4d44-4486-8832-cb7cee546fb4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:15:18.541518 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbc8a2a-4d44-4486-8832-cb7cee546fb4 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:15:18.589946 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 577/1173] 10.222.0.87 () {60 vars in 1141 bytes} [Tue Jun 21 18:15:18 2022] POST /identity/v3/projects => generated 361 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:15:21.574864 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de36bdd-8b61-4486-8ead-5341d822531b None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:21.575458 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de36bdd-8b61-4486-8ead-5341d822531b None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:21.575866 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de36bdd-8b61-4486-8ead-5341d822531b None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:21.578534 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 597/1174] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:15:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:15:21.588256 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32250e76-248c-40f9-a843-376d45329d39 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:21.589094 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32250e76-248c-40f9-a843-376d45329d39 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:21.589266 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32250e76-248c-40f9-a843-376d45329d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:21.616092 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-32250e76-248c-40f9-a843-376d45329d39 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:15:21.618376 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-32250e76-248c-40f9-a843-376d45329d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:15:21.634184 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:15:21.734792 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 578/1175] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:15:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:15:21.765915 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de36bdd-8b61-4486-8ead-5341d822531b None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:15:21.933945 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de36bdd-8b61-4486-8ead-5341d822531b None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:15:21.987212 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0328d9-648a-4e34-ba61-f153bf4f033c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:15:21.988176 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0328d9-648a-4e34-ba61-f153bf4f033c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:21.988501 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0328d9-648a-4e34-ba61-f153bf4f033c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:21.988829 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0328d9-648a-4e34-ba61-f153bf4f033c None admin] PATH_INFO: `/v3/domains/f06c4fe3b22e4023919df0dff134c6dc` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:21.990403 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0328d9-648a-4e34-ba61-f153bf4f033c None admin] RBAC: Authorizing `identity:get_domain(domain_id=f06c4fe3b22e4023919df0dff134c6dc)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:15:21.991978 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0328d9-648a-4e34-ba61-f153bf4f033c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:15:21.994133 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 598/1176] 10.222.0.87 () {56 vars in 1183 bytes} [Tue Jun 21 18:15:21 2022] GET /identity/v3/domains/f06c4fe3b22e4023919df0dff134c6dc => generated 267 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:15:22.001847 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32250e76-248c-40f9-a843-376d45329d39 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:15:22.185838 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32250e76-248c-40f9-a843-376d45329d39 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:15:22.250033 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d58035-2798-4524-99b2-5a64ebf8bd18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:15:22.251278 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d58035-2798-4524-99b2-5a64ebf8bd18 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:22.251802 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d58035-2798-4524-99b2-5a64ebf8bd18 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:22.252313 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d58035-2798-4524-99b2-5a64ebf8bd18 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:22.253764 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d58035-2798-4524-99b2-5a64ebf8bd18 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:15:22.257000 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d58035-2798-4524-99b2-5a64ebf8bd18 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:15:22.282741 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:15:22.295525 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 579/1177] 10.222.0.87 () {60 vars in 1132 bytes} [Tue Jun 21 18:15:22 2022] POST /identity/v3/users => generated 311 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:15:25.341185 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d7b50-1df0-4ad0-833f-0433a5db6d01 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:25.341804 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d7b50-1df0-4ad0-833f-0433a5db6d01 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:25.342195 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d7b50-1df0-4ad0-833f-0433a5db6d01 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:25.344364 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 599/1178] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:15:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:15:25.353855 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b98b643-06c5-4045-86e3-c091d72ff65e None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:25.354807 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b98b643-06c5-4045-86e3-c091d72ff65e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:25.354807 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b98b643-06c5-4045-86e3-c091d72ff65e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:25.376510 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-8b98b643-06c5-4045-86e3-c091d72ff65e None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:15:25.378036 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8b98b643-06c5-4045-86e3-c091d72ff65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:15:25.483716 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 580/1179] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:15:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:15:25.510483 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d7b50-1df0-4ad0-833f-0433a5db6d01 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:15:25.571858 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:15:25.681179 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d7b50-1df0-4ad0-833f-0433a5db6d01 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:15:25.729666 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6774ced5-a6d9-46e3-8f7f-69bcf55b3406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:15:25.730604 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774ced5-a6d9-46e3-8f7f-69bcf55b3406 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:25.730956 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774ced5-a6d9-46e3-8f7f-69bcf55b3406 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:25.731340 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774ced5-a6d9-46e3-8f7f-69bcf55b3406 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:25.749061 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6774ced5-a6d9-46e3-8f7f-69bcf55b3406 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:15:25.760678 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-6774ced5-a6d9-46e3-8f7f-69bcf55b3406 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:15:25.760678 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:15:25.760678 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:15:25.760678 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:15:25.760678 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:15:25.760678 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:15:25.760678 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:15:25.760678 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:15:25.760678 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:15:25.760678 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:15:25.760678 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:15:25.760678 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:15:25.760678 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:15:25.760678 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:15:25.760678 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:15:25.760678 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:15:25.762481 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:15:25.762481 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:15:25.762481 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:15:25.762481 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:15:25.762481 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:15:25.762481 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:15:25.762481 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:15:25.762481 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:15:25.762481 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:15:25.762481 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:15:25.762481 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:15:25.762481 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:15:25.762481 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:15:25.762481 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:15:25.762481 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:15:25.762481 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:15:25.763530 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:15:25.763530 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:15:25.763530 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:15:25.763530 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:15:25.763530 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:15:25.763530 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:15:25.763530 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:15:25.763530 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:15:25.763530 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:15:25.763530 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:15:25.763530 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:15:25.763530 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:15:25.763530 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 600/1180] 10.222.0.87 () {56 vars in 1096 bytes} [Tue Jun 21 18:15:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:15:25.769931 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b98b643-06c5-4045-86e3-c091d72ff65e None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:15:25.931641 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b98b643-06c5-4045-86e3-c091d72ff65e None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:15:25.983162 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:15:25.996770 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55ce174-d920-4bf3-a134-dd377a9a2df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:15:25.997932 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ce174-d920-4bf3-a134-dd377a9a2df4 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:25.998345 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ce174-d920-4bf3-a134-dd377a9a2df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:25.998799 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ce174-d920-4bf3-a134-dd377a9a2df4 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:25.999716 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55ce174-d920-4bf3-a134-dd377a9a2df4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:15:26.000046 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55ce174-d920-4bf3-a134-dd377a9a2df4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:15:26.001531 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55ce174-d920-4bf3-a134-dd377a9a2df4 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:15:26.011983 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 581/1181] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:15:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:15:26.022062 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6774ced5-a6d9-46e3-8f7f-69bcf55b3406 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:15:26.164897 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:15:26.176796 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6774ced5-a6d9-46e3-8f7f-69bcf55b3406 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:15:26.236825 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bb58f1-61b2-4786-b421-86080a9cbe23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:15:26.237676 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb58f1-61b2-4786-b421-86080a9cbe23 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:26.237960 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb58f1-61b2-4786-b421-86080a9cbe23 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:26.238240 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb58f1-61b2-4786-b421-86080a9cbe23 None admin] PATH_INFO: `/v3/users/36673694926d4e39b1dd99d86e60bc13` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:26.255301 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bb58f1-61b2-4786-b421-86080a9cbe23 None admin] RBAC: Authorizing `identity:get_user(user_id=36673694926d4e39b1dd99d86e60bc13)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:15:26.256883 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bb58f1-61b2-4786-b421-86080a9cbe23 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:15:26.259021 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 601/1182] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:15:26 2022] GET /identity/v3/users/36673694926d4e39b1dd99d86e60bc13 => generated 311 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:15:26.266995 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55ce174-d920-4bf3-a134-dd377a9a2df4 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:15:26.444062 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55ce174-d920-4bf3-a134-dd377a9a2df4 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:15:26.492678 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b28263-0b1b-4d27-a7d6-0aaae0823e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:15:26.493385 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b28263-0b1b-4d27-a7d6-0aaae0823e17 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:26.493714 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b28263-0b1b-4d27-a7d6-0aaae0823e17 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:26.493998 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b28263-0b1b-4d27-a7d6-0aaae0823e17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:26.589456 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:15:26.608514 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b28263-0b1b-4d27-a7d6-0aaae0823e17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:15:26.610499 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b28263-0b1b-4d27-a7d6-0aaae0823e17 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:15:26.752595 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 582/1183] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:15:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:15:26.759230 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bb58f1-61b2-4786-b421-86080a9cbe23 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:15:26.908944 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bb58f1-61b2-4786-b421-86080a9cbe23 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:15:26.951820 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c56af5a-7419-4a12-8d0e-4a65a70b92c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:15:26.952715 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56af5a-7419-4a12-8d0e-4a65a70b92c7 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:26.953002 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56af5a-7419-4a12-8d0e-4a65a70b92c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:26.953274 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56af5a-7419-4a12-8d0e-4a65a70b92c7 None admin] PATH_INFO: `/v3/projects/de03bb6191414c05abf533ab7534868a` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:26.954708 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c56af5a-7419-4a12-8d0e-4a65a70b92c7 None admin] RBAC: Authorizing `identity:get_project(project_id=de03bb6191414c05abf533ab7534868a)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:15:26.956213 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c56af5a-7419-4a12-8d0e-4a65a70b92c7 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:15:26.958144 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 602/1184] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:15:26 2022] GET /identity/v3/projects/de03bb6191414c05abf533ab7534868a => generated 361 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:15:26.965185 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b28263-0b1b-4d27-a7d6-0aaae0823e17 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:15:27.107613 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:15:27.128255 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b28263-0b1b-4d27-a7d6-0aaae0823e17 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:15:27.184088 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1dde0f-8d80-4486-95b7-bb723bb11ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:15:27.185285 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1dde0f-8d80-4486-95b7-bb723bb11ad5 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:27.185826 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1dde0f-8d80-4486-95b7-bb723bb11ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:27.186326 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1dde0f-8d80-4486-95b7-bb723bb11ad5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:27.187941 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1dde0f-8d80-4486-95b7-bb723bb11ad5 None admin] RBAC: Adding query filter params (role.id=98f0943a610f4a9e9f8de54f6e9f717d, scope.project.id=de03bb6191414c05abf533ab7534868a, user.id=36673694926d4e39b1dd99d86e60bc13) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:15:27.188504 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1dde0f-8d80-4486-95b7-bb723bb11ad5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:15:27.191389 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1dde0f-8d80-4486-95b7-bb723bb11ad5 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:15:27.203804 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 583/1185] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:15:26 2022] GET /identity/v3/role_assignments?user.id=36673694926d4e39b1dd99d86e60bc13&scope.project.id=de03bb6191414c05abf533ab7534868a&role.id=98f0943a610f4a9e9f8de54f6e9f717d => generated 259 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:15:29.844092 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bcb79-c167-40fa-96a9-4628099ceef7 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:29.844868 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bcb79-c167-40fa-96a9-4628099ceef7 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:29.845083 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bcb79-c167-40fa-96a9-4628099ceef7 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:29.848163 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 603/1186] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:15:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:15:29.859126 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20cdfe7-8413-44d8-a9c1-63e1178be4b4 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:29.859812 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20cdfe7-8413-44d8-a9c1-63e1178be4b4 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:29.859999 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20cdfe7-8413-44d8-a9c1-63e1178be4b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:29.890134 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f20cdfe7-8413-44d8-a9c1-63e1178be4b4 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:15:29.892301 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f20cdfe7-8413-44d8-a9c1-63e1178be4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:15:30.004898 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 584/1187] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:15:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:15:30.042559 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4bcb79-c167-40fa-96a9-4628099ceef7 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:15:30.081013 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:15:30.251753 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4bcb79-c167-40fa-96a9-4628099ceef7 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:15:30.301909 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dfba4a-d221-424d-9687-4028146b99b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:15:30.302735 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfba4a-d221-424d-9687-4028146b99b8 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:30.303018 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfba4a-d221-424d-9687-4028146b99b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:30.303299 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfba4a-d221-424d-9687-4028146b99b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:30.320305 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dfba4a-d221-424d-9687-4028146b99b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:15:30.328425 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-77dfba4a-d221-424d-9687-4028146b99b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:15:30.328425 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:15:30.328425 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:15:30.328425 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:15:30.328425 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:15:30.328425 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:15:30.328425 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:15:30.328425 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:15:30.328425 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:15:30.328425 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:15:30.328425 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:15:30.328425 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:15:30.328425 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:15:30.328425 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:15:30.328425 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:15:30.328425 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:15:30.330278 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:15:30.330278 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:15:30.330278 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:15:30.330278 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:15:30.330278 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:15:30.330278 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:15:30.330278 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:15:30.330278 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:15:30.330278 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:15:30.330278 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:15:30.330278 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:15:30.330278 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:15:30.330278 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:15:30.330278 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:15:30.330278 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:15:30.330278 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:15:30.331972 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:15:30.331972 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:15:30.331972 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:15:30.331972 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:15:30.331972 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:15:30.331972 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:15:30.331972 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:15:30.331972 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:15:30.331972 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:15:30.331972 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:15:30.331972 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:15:30.331972 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:15:30.331972 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 604/1188] 10.222.0.87 () {56 vars in 1096 bytes} [Tue Jun 21 18:15:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:15:30.338544 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20cdfe7-8413-44d8-a9c1-63e1178be4b4 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:15:30.479413 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:15:30.508453 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20cdfe7-8413-44d8-a9c1-63e1178be4b4 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:15:30.577847 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0412c3b8-6ba0-4236-9d5e-06d227509a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:15:30.579209 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412c3b8-6ba0-4236-9d5e-06d227509a25 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:30.579703 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412c3b8-6ba0-4236-9d5e-06d227509a25 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:30.580217 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412c3b8-6ba0-4236-9d5e-06d227509a25 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:30.581883 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0412c3b8-6ba0-4236-9d5e-06d227509a25 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:15:30.582555 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0412c3b8-6ba0-4236-9d5e-06d227509a25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:15:30.585240 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0412c3b8-6ba0-4236-9d5e-06d227509a25 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:15:30.603567 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 585/1189] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:15:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:15:30.613690 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dfba4a-d221-424d-9687-4028146b99b8 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:15:30.761761 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:15:30.810345 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dfba4a-d221-424d-9687-4028146b99b8 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:15:30.885623 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c7125-1165-4e67-b527-2292d2a176a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:15:30.887206 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c7125-1165-4e67-b527-2292d2a176a3 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:30.887681 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c7125-1165-4e67-b527-2292d2a176a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:30.888163 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c7125-1165-4e67-b527-2292d2a176a3 None admin] PATH_INFO: `/v3/users/36673694926d4e39b1dd99d86e60bc13` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:30.899175 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901c7125-1165-4e67-b527-2292d2a176a3 None admin] RBAC: Authorizing `identity:get_user(user_id=36673694926d4e39b1dd99d86e60bc13)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:15:30.902235 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901c7125-1165-4e67-b527-2292d2a176a3 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:15:30.905592 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 605/1190] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:15:30 2022] GET /identity/v3/users/36673694926d4e39b1dd99d86e60bc13 => generated 311 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:15:30.917597 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0412c3b8-6ba0-4236-9d5e-06d227509a25 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:15:31.077136 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0412c3b8-6ba0-4236-9d5e-06d227509a25 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:15:31.125347 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d7baa1-898a-4e8b-89bc-756fc7a54c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:15:31.126307 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7baa1-898a-4e8b-89bc-756fc7a54c61 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:31.126589 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7baa1-898a-4e8b-89bc-756fc7a54c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:31.126867 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7baa1-898a-4e8b-89bc-756fc7a54c61 None admin] PATH_INFO: `/v3/projects/de03bb6191414c05abf533ab7534868a` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:31.128181 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d7baa1-898a-4e8b-89bc-756fc7a54c61 None admin] RBAC: Authorizing `identity:get_project(project_id=de03bb6191414c05abf533ab7534868a)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:15:31.129775 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d7baa1-898a-4e8b-89bc-756fc7a54c61 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:15:31.131671 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 586/1191] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:15:30 2022] GET /identity/v3/projects/de03bb6191414c05abf533ab7534868a => generated 361 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:15:31.138625 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c7125-1165-4e67-b527-2292d2a176a3 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:15:31.274471 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c7125-1165-4e67-b527-2292d2a176a3 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:15:31.330530 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea918ce2-b37e-4430-bfbb-0752d258680e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:15:31.331633 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea918ce2-b37e-4430-bfbb-0752d258680e None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:31.331940 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea918ce2-b37e-4430-bfbb-0752d258680e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:31.332418 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea918ce2-b37e-4430-bfbb-0752d258680e None admin] PATH_INFO: `/v3/projects/de03bb6191414c05abf533ab7534868a/users/36673694926d4e39b1dd99d86e60bc13/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:31.335107 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea918ce2-b37e-4430-bfbb-0752d258680e None admin] RBAC: Authorizing `identity:create_grant(project_id=de03bb6191414c05abf533ab7534868a, user_id=36673694926d4e39b1dd99d86e60bc13, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:15:31.337251 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea918ce2-b37e-4430-bfbb-0752d258680e None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:15:31.358156 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 606/1192] 10.222.0.87 () {58 vars in 1439 bytes} [Tue Jun 21 18:15:31 2022] PUT /identity/v3/projects/de03bb6191414c05abf533ab7534868a/users/36673694926d4e39b1dd99d86e60bc13/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:15:33.764870 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1021a731-421d-44d4-af8d-829b0cc53fd7 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:33.765788 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1021a731-421d-44d4-af8d-829b0cc53fd7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:33.765954 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1021a731-421d-44d4-af8d-829b0cc53fd7 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:33.768426 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 587/1193] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:15:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:15:33.775872 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669fdb0c-7238-44c7-867b-b0f9c1dd8ce8 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:33.776527 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669fdb0c-7238-44c7-867b-b0f9c1dd8ce8 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:33.776952 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669fdb0c-7238-44c7-867b-b0f9c1dd8ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:33.799431 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:15:33.809310 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-669fdb0c-7238-44c7-867b-b0f9c1dd8ce8 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:15:33.811513 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-669fdb0c-7238-44c7-867b-b0f9c1dd8ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:15:33.946735 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 607/1194] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:15:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:15:33.980691 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1021a731-421d-44d4-af8d-829b0cc53fd7 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:15:34.006867 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:15:34.132413 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1021a731-421d-44d4-af8d-829b0cc53fd7 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:15:34.180183 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11a548d-781e-44b2-b63f-5b2adafe4d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:15:34.181159 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a548d-781e-44b2-b63f-5b2adafe4d3a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:34.181492 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a548d-781e-44b2-b63f-5b2adafe4d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:34.181896 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a548d-781e-44b2-b63f-5b2adafe4d3a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:34.195139 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11a548d-781e-44b2-b63f-5b2adafe4d3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:15:34.203412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-b11a548d-781e-44b2-b63f-5b2adafe4d3a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:15:34.203412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:15:34.203412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:15:34.203412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:15:34.203412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:15:34.203412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:15:34.203412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:15:34.203412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:15:34.203412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:15:34.203412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:15:34.203412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:15:34.203412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:15:34.203412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:15:34.203412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:15:34.203412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:15:34.203412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:15:34.205582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:15:34.205582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:15:34.205582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:15:34.205582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:15:34.205582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:15:34.205582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:15:34.205582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:15:34.205582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:15:34.205582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:15:34.205582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:15:34.205582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:15:34.205582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:15:34.205582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:15:34.205582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:15:34.205582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:15:34.205582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:15:34.207651 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:15:34.207651 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:15:34.207651 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:15:34.207651 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:15:34.207651 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:15:34.207651 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:15:34.207651 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:15:34.207651 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:15:34.207651 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:15:34.207651 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:15:34.207651 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:15:34.207651 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:15:34.207651 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 588/1195] 10.222.0.87 () {56 vars in 1096 bytes} [Tue Jun 21 18:15:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:15:34.215422 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669fdb0c-7238-44c7-867b-b0f9c1dd8ce8 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:15:34.340610 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669fdb0c-7238-44c7-867b-b0f9c1dd8ce8 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:15:34.407001 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac16492-90a9-4707-ba58-325861c7511e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:15:34.408091 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac16492-90a9-4707-ba58-325861c7511e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:34.408499 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac16492-90a9-4707-ba58-325861c7511e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:34.408983 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac16492-90a9-4707-ba58-325861c7511e None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:34.410472 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac16492-90a9-4707-ba58-325861c7511e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:15:34.411013 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac16492-90a9-4707-ba58-325861c7511e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:15:34.413685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac16492-90a9-4707-ba58-325861c7511e None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:15:34.418465 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:15:34.435313 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 608/1196] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:15:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:15:34.447203 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11a548d-781e-44b2-b63f-5b2adafe4d3a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:15:34.566982 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:15:34.616840 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11a548d-781e-44b2-b63f-5b2adafe4d3a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:15:34.687424 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880d1764-da08-47f8-ad49-ab359a0f43ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:15:34.688989 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d1764-da08-47f8-ad49-ab359a0f43ea None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:34.689481 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d1764-da08-47f8-ad49-ab359a0f43ea None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:34.690005 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d1764-da08-47f8-ad49-ab359a0f43ea None admin] PATH_INFO: `/v3/users/36673694926d4e39b1dd99d86e60bc13` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:34.693374 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880d1764-da08-47f8-ad49-ab359a0f43ea None admin] RBAC: Authorizing `identity:get_user(user_id=36673694926d4e39b1dd99d86e60bc13)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:15:34.696554 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880d1764-da08-47f8-ad49-ab359a0f43ea None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:15:34.701068 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 589/1197] 10.222.0.87 () {56 vars in 1177 bytes} [Tue Jun 21 18:15:34 2022] GET /identity/v3/users/36673694926d4e39b1dd99d86e60bc13 => generated 311 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:15:34.710787 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac16492-90a9-4707-ba58-325861c7511e None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:15:34.883402 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac16492-90a9-4707-ba58-325861c7511e None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:15:34.927757 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8e02c7-6b00-42e7-8342-c6a28855f94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:15:34.928560 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e02c7-6b00-42e7-8342-c6a28855f94a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:34.928936 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e02c7-6b00-42e7-8342-c6a28855f94a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:34.929303 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e02c7-6b00-42e7-8342-c6a28855f94a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:35.024755 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:15:35.031669 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8e02c7-6b00-42e7-8342-c6a28855f94a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:15:35.033374 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8e02c7-6b00-42e7-8342-c6a28855f94a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:15:35.230963 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 609/1198] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:15:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 21 18:15:35.238523 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880d1764-da08-47f8-ad49-ab359a0f43ea None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:15:35.407348 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880d1764-da08-47f8-ad49-ab359a0f43ea None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:15:35.468401 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31058ca-9115-4bb3-b72e-6f579bd3fac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:15:35.469920 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31058ca-9115-4bb3-b72e-6f579bd3fac6 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:35.470339 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31058ca-9115-4bb3-b72e-6f579bd3fac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:35.470756 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31058ca-9115-4bb3-b72e-6f579bd3fac6 None admin] PATH_INFO: `/v3/projects/de03bb6191414c05abf533ab7534868a` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:35.473149 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31058ca-9115-4bb3-b72e-6f579bd3fac6 None admin] RBAC: Authorizing `identity:get_project(project_id=de03bb6191414c05abf533ab7534868a)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:15:35.476345 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31058ca-9115-4bb3-b72e-6f579bd3fac6 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:15:35.479923 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 590/1199] 10.222.0.87 () {56 vars in 1186 bytes} [Tue Jun 21 18:15:35 2022] GET /identity/v3/projects/de03bb6191414c05abf533ab7534868a => generated 361 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:15:35.490794 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8e02c7-6b00-42e7-8342-c6a28855f94a None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:15:35.635319 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:15:35.643506 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8e02c7-6b00-42e7-8342-c6a28855f94a None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:15:35.718439 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288e3825-2eac-4da5-9ded-fb1b379d7d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:15:35.719597 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e3825-2eac-4da5-9ded-fb1b379d7d45 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:15:35.720088 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e3825-2eac-4da5-9ded-fb1b379d7d45 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:15:35.720574 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e3825-2eac-4da5-9ded-fb1b379d7d45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:15:35.722293 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288e3825-2eac-4da5-9ded-fb1b379d7d45 None admin] RBAC: Adding query filter params (role.id=98f0943a610f4a9e9f8de54f6e9f717d, scope.project.id=de03bb6191414c05abf533ab7534868a, user.id=36673694926d4e39b1dd99d86e60bc13) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:15:35.722854 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288e3825-2eac-4da5-9ded-fb1b379d7d45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:15:35.726042 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288e3825-2eac-4da5-9ded-fb1b379d7d45 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:15:35.738759 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 610/1200] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:15:35 2022] GET /identity/v3/role_assignments?user.id=36673694926d4e39b1dd99d86e60bc13&scope.project.id=de03bb6191414c05abf533ab7534868a&role.id=98f0943a610f4a9e9f8de54f6e9f717d => generated 608 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:15:45.460117 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:15:45.732740 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=19 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:16:53.581687 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae3b01-018f-4711-8fd7-bafa6c74d70a None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:16:53.582645 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae3b01-018f-4711-8fd7-bafa6c74d70a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:16:53.582645 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae3b01-018f-4711-8fd7-bafa6c74d70a None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:16:53.584296 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 591/1201] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:16:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:16:53.593418 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13d40c-4251-420c-9df1-154121465794 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:16:53.594403 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13d40c-4251-420c-9df1-154121465794 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:16:53.594403 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13d40c-4251-420c-9df1-154121465794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:16:53.625769 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-5e13d40c-4251-420c-9df1-154121465794 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:16:53.627161 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5e13d40c-4251-420c-9df1-154121465794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:16:53.855668 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 611/1202] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:16:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:16:53.899187 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ae3b01-018f-4711-8fd7-bafa6c74d70a None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:16:54.209536 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ae3b01-018f-4711-8fd7-bafa6c74d70a None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:16:54.332107 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd631b1d-7293-4311-a23f-a624491306a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:16:54.333619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd631b1d-7293-4311-a23f-a624491306a8 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:16:54.334220 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd631b1d-7293-4311-a23f-a624491306a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:16:54.335646 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd631b1d-7293-4311-a23f-a624491306a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:16:54.372019 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd631b1d-7293-4311-a23f-a624491306a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:16:54.374970 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd631b1d-7293-4311-a23f-a624491306a8 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:16:54.411588 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-cd631b1d-7293-4311-a23f-a624491306a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:16:54.411588 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:16:54.411588 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:16:54.411588 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:16:54.411588 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:16:54.411588 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:16:54.411588 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:16:54.411588 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:16:54.411588 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:16:54.411588 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:16:54.411588 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:16:54.411588 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:16:54.411588 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:16:54.411588 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:16:54.411588 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:16:54.413582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:16:54.413582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:16:54.413582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:16:54.413582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:16:54.413582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:16:54.413582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:16:54.413582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:16:54.413582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:16:54.413582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:16:54.413582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:16:54.413582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:16:54.413582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:16:54.413582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:16:54.413582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:16:54.413582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:16:54.413582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:16:54.413582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:16:54.416397 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:16:54.416397 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:16:54.416397 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:16:54.416397 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:16:54.421986 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 592/1203] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:16:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 524 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:16:54.422886 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13d40c-4251-420c-9df1-154121465794 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:16:54.710310 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13d40c-4251-420c-9df1-154121465794 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:16:54.808040 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:16:54.819930 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fcc95b-f612-45fb-8415-0fc4d1313122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:16:54.822528 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcc95b-f612-45fb-8415-0fc4d1313122 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:16:54.822528 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcc95b-f612-45fb-8415-0fc4d1313122 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:16:54.822528 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcc95b-f612-45fb-8415-0fc4d1313122 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:16:54.822528 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fcc95b-f612-45fb-8415-0fc4d1313122 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:16:54.823039 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fcc95b-f612-45fb-8415-0fc4d1313122 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:16:54.824466 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fcc95b-f612-45fb-8415-0fc4d1313122 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:16:54.863688 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 612/1204] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:16:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 438 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:16:54.870485 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd631b1d-7293-4311-a23f-a624491306a8 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:16:55.059299 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:16:55.223656 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd631b1d-7293-4311-a23f-a624491306a8 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:16:55.350519 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d45de-4de7-46a5-be89-a6ad21d90eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:16:55.351903 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d45de-4de7-46a5-be89-a6ad21d90eaf None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:16:55.352433 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d45de-4de7-46a5-be89-a6ad21d90eaf None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:16:55.352934 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d45de-4de7-46a5-be89-a6ad21d90eaf None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:16:55.354390 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07d45de-4de7-46a5-be89-a6ad21d90eaf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:16:55.357537 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07d45de-4de7-46a5-be89-a6ad21d90eaf None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:16:55.407882 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 593/1205] 10.222.0.87 () {60 vars in 1132 bytes} [Tue Jun 21 18:16:54 2022] POST /identity/v3/users => generated 251 bytes in 538 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:16:58.936331 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e5f6b7-22c0-443c-9f27-6df2207d7a1b None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:16:58.937096 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e5f6b7-22c0-443c-9f27-6df2207d7a1b None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:16:58.937096 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e5f6b7-22c0-443c-9f27-6df2207d7a1b None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:16:58.940227 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 613/1206] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:16:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:16:58.948671 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60104a33-1789-42e7-8338-563b0e9bbf13 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:16:58.949240 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60104a33-1789-42e7-8338-563b0e9bbf13 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:16:58.949707 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60104a33-1789-42e7-8338-563b0e9bbf13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:16:58.975632 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-60104a33-1789-42e7-8338-563b0e9bbf13 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:16:58.977189 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-60104a33-1789-42e7-8338-563b0e9bbf13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:16:59.153678 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 594/1207] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:16:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:16:59.203379 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e5f6b7-22c0-443c-9f27-6df2207d7a1b None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:16:59.464314 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e5f6b7-22c0-443c-9f27-6df2207d7a1b None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:16:59.533516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0788b71c-74b7-494f-899d-e25cda60214e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:16:59.535045 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0788b71c-74b7-494f-899d-e25cda60214e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:16:59.535520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0788b71c-74b7-494f-899d-e25cda60214e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:16:59.536025 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0788b71c-74b7-494f-899d-e25cda60214e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:16:59.564412 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0788b71c-74b7-494f-899d-e25cda60214e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:16:59.579964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-0788b71c-74b7-494f-899d-e25cda60214e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:16:59.579964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:16:59.579964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:16:59.579964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:16:59.579964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:16:59.579964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:16:59.579964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:16:59.579964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:16:59.579964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:16:59.579964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:16:59.579964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:16:59.579964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:16:59.579964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:16:59.579964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:16:59.579964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:16:59.579964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:16:59.581800 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:16:59.581800 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:16:59.581800 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:16:59.581800 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:16:59.581800 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:16:59.581800 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:16:59.581800 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:16:59.581800 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:16:59.581800 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:16:59.581800 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:16:59.581800 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:16:59.581800 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:16:59.581800 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:16:59.581800 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:16:59.581800 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:16:59.581800 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:16:59.583024 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:16:59.583024 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:16:59.583024 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:16:59.583024 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:16:59.583024 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:16:59.583024 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:16:59.583024 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:16:59.583024 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:16:59.583024 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:16:59.583024 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:16:59.583024 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:16:59.583024 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:16:59.583024 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 614/1208] 10.222.0.87 () {56 vars in 1102 bytes} [Tue Jun 21 18:16:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:16:59.594970 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60104a33-1789-42e7-8338-563b0e9bbf13 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:16:59.717608 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:16:59.896368 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60104a33-1789-42e7-8338-563b0e9bbf13 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:16:59.978120 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e32b25-63cb-48cf-ba28-3b6a81f0d5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:16:59.979184 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e32b25-63cb-48cf-ba28-3b6a81f0d5b9 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:16:59.979629 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e32b25-63cb-48cf-ba28-3b6a81f0d5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:16:59.980108 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e32b25-63cb-48cf-ba28-3b6a81f0d5b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:16:59.981534 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e32b25-63cb-48cf-ba28-3b6a81f0d5b9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:16:59.982124 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e32b25-63cb-48cf-ba28-3b6a81f0d5b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:16:59.984885 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e32b25-63cb-48cf-ba28-3b6a81f0d5b9 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:00.007366 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 595/1209] 10.222.0.87 () {56 vars in 1103 bytes} [Tue Jun 21 18:16:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 414 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:17:00.018188 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0788b71c-74b7-494f-899d-e25cda60214e None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:00.141277 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:00.260934 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0788b71c-74b7-494f-899d-e25cda60214e None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:00.349672 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5340fbdb-d366-4b81-9341-06d681ca9638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:00.351382 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5340fbdb-d366-4b81-9341-06d681ca9638 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:00.355603 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5340fbdb-d366-4b81-9341-06d681ca9638 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:00.359792 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5340fbdb-d366-4b81-9341-06d681ca9638 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:00.391389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5340fbdb-d366-4b81-9341-06d681ca9638 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:00.394329 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5340fbdb-d366-4b81-9341-06d681ca9638 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:00.446930 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-5340fbdb-d366-4b81-9341-06d681ca9638 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:17:00.446930 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:17:00.446930 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:17:00.446930 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:17:00.446930 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:17:00.446930 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:17:00.446930 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:17:00.446930 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:17:00.446930 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:17:00.446930 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:17:00.446930 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:17:00.446930 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:17:00.446930 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:17:00.446930 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:17:00.446930 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:17:00.448873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:17:00.448873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:17:00.448873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:17:00.448873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:17:00.448873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:17:00.448873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:17:00.448873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:17:00.448873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:17:00.448873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:17:00.448873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:17:00.448873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:17:00.448873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:17:00.448873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:17:00.448873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:17:00.448873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:17:00.448873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:17:00.448873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:17:00.451781 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:17:00.451781 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:17:00.451781 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:17:00.451781 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:17:00.451781 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 615/1210] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:17:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 433 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:17:00.460887 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e32b25-63cb-48cf-ba28-3b6a81f0d5b9 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:00.706798 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:00.714916 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e32b25-63cb-48cf-ba28-3b6a81f0d5b9 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:00.805712 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bacac3f-f888-4187-91e1-c50dda95efe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:00.806854 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bacac3f-f888-4187-91e1-c50dda95efe9 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:00.807419 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bacac3f-f888-4187-91e1-c50dda95efe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:00.808443 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bacac3f-f888-4187-91e1-c50dda95efe9 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:00.809514 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bacac3f-f888-4187-91e1-c50dda95efe9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:17:00.810080 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bacac3f-f888-4187-91e1-c50dda95efe9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:00.813264 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bacac3f-f888-4187-91e1-c50dda95efe9 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:00.848046 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 596/1211] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:17:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:17:00.850255 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5340fbdb-d366-4b81-9341-06d681ca9638 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:01.061279 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5340fbdb-d366-4b81-9341-06d681ca9638 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:01.064344 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:01.132523 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eb3417-05c3-46d8-b34e-8b689edce67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:01.134767 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eb3417-05c3-46d8-b34e-8b689edce67d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:01.135098 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eb3417-05c3-46d8-b34e-8b689edce67d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:01.135530 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eb3417-05c3-46d8-b34e-8b689edce67d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:01.147885 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eb3417-05c3-46d8-b34e-8b689edce67d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:01.151079 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eb3417-05c3-46d8-b34e-8b689edce67d None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:01.182054 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-e6eb3417-05c3-46d8-b34e-8b689edce67d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 21 18:17:01.182054 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:17:01.182054 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:17:01.182054 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:17:01.182054 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:17:01.182054 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:17:01.182054 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:17:01.182054 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:17:01.182054 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:17:01.182054 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:17:01.182054 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:17:01.182054 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:17:01.182054 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:17:01.182054 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:17:01.182054 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:17:01.182054 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:17:01.193791 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:17:01.193791 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:17:01.193791 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:17:01.193791 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:17:01.193791 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:17:01.193791 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:17:01.193791 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:17:01.193791 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:17:01.193791 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:17:01.193791 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:17:01.193791 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:17:01.193791 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:17:01.193791 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:17:01.193791 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:17:01.193791 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:17:01.193791 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:17:01.193791 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:17:01.193791 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:17:01.196600 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:17:01.196600 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:17:01.196600 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:17:01.196600 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:17:01.196600 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:17:01.196600 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:17:01.196600 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:17:01.196600 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:17:01.196600 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 21 18:17:01.196600 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:17:01.196600 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 616/1212] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:17:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:17:01.197833 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bacac3f-f888-4187-91e1-c50dda95efe9 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:01.407694 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bacac3f-f888-4187-91e1-c50dda95efe9 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:01.491385 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eee87b8-f6d0-4cd4-8390-0245c6fa219c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:01.492226 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee87b8-f6d0-4cd4-8390-0245c6fa219c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:01.492534 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee87b8-f6d0-4cd4-8390-0245c6fa219c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:01.492812 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee87b8-f6d0-4cd4-8390-0245c6fa219c None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:01.493984 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eee87b8-f6d0-4cd4-8390-0245c6fa219c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:17:01.494357 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eee87b8-f6d0-4cd4-8390-0245c6fa219c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:01.497548 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eee87b8-f6d0-4cd4-8390-0245c6fa219c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:01.533420 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 597/1213] 10.222.0.87 () {56 vars in 1143 bytes} [Tue Jun 21 18:17:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:17:01.551020 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eb3417-05c3-46d8-b34e-8b689edce67d None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:01.817811 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eb3417-05c3-46d8-b34e-8b689edce67d None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:01.914834 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e98cd2-1fc4-45a5-ad81-e674d39417cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:01.915956 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e98cd2-1fc4-45a5-ad81-e674d39417cf None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:01.916452 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e98cd2-1fc4-45a5-ad81-e674d39417cf None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:01.917300 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e98cd2-1fc4-45a5-ad81-e674d39417cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:02.003423 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:02.097139 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e98cd2-1fc4-45a5-ad81-e674d39417cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:02.100801 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e98cd2-1fc4-45a5-ad81-e674d39417cf None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:02.343736 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 617/1214] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:17:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 789 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:17:02.350589 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eee87b8-f6d0-4cd4-8390-0245c6fa219c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:02.392831 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:02.583849 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eee87b8-f6d0-4cd4-8390-0245c6fa219c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:02.667895 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d8381-ea7e-4b57-8fbd-74118b4e389b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:02.669229 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d8381-ea7e-4b57-8fbd-74118b4e389b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:02.669983 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d8381-ea7e-4b57-8fbd-74118b4e389b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:02.669983 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d8381-ea7e-4b57-8fbd-74118b4e389b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:02.692285 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337d8381-ea7e-4b57-8fbd-74118b4e389b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:02.695222 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337d8381-ea7e-4b57-8fbd-74118b4e389b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:02.739779 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-337d8381-ea7e-4b57-8fbd-74118b4e389b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:17:02.739779 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:17:02.739779 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:17:02.739779 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:17:02.739779 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:17:02.739779 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:17:02.739779 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:17:02.739779 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:17:02.739779 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:17:02.739779 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:17:02.739779 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:17:02.739779 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:17:02.739779 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:17:02.739779 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:17:02.739779 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:17:02.741666 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:17:02.741666 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:17:02.741666 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:17:02.741666 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:17:02.741666 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:17:02.741666 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:17:02.741666 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:17:02.741666 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:17:02.741666 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:17:02.741666 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:17:02.741666 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:17:02.741666 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:17:02.741666 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:17:02.741666 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:17:02.741666 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:17:02.741666 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:17:02.741666 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:17:02.743580 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:17:02.743580 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:17:02.743580 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:17:02.743580 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:17:02.743580 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 598/1215] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:17:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 393 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:17:02.752969 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e98cd2-1fc4-45a5-ad81-e674d39417cf None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:02.921099 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:03.004400 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e98cd2-1fc4-45a5-ad81-e674d39417cf None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:03.085041 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a35e149-19c3-49cd-a540-0eee4d2794cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:03.086120 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35e149-19c3-49cd-a540-0eee4d2794cb None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:03.086437 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35e149-19c3-49cd-a540-0eee4d2794cb None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:03.086744 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35e149-19c3-49cd-a540-0eee4d2794cb None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:03.088034 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a35e149-19c3-49cd-a540-0eee4d2794cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:17:03.088605 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a35e149-19c3-49cd-a540-0eee4d2794cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:03.091109 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a35e149-19c3-49cd-a540-0eee4d2794cb None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:03.122041 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 618/1216] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:17:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:17:03.129376 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d8381-ea7e-4b57-8fbd-74118b4e389b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:03.283168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:03.366537 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d8381-ea7e-4b57-8fbd-74118b4e389b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:03.450793 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22537b23-3401-4fe6-b7d1-ec80283c0d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:03.452122 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22537b23-3401-4fe6-b7d1-ec80283c0d8c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:03.452633 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22537b23-3401-4fe6-b7d1-ec80283c0d8c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:03.453150 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22537b23-3401-4fe6-b7d1-ec80283c0d8c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:03.469934 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22537b23-3401-4fe6-b7d1-ec80283c0d8c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:03.472881 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22537b23-3401-4fe6-b7d1-ec80283c0d8c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:03.512142 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-22537b23-3401-4fe6-b7d1-ec80283c0d8c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:17:03.512142 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:17:03.512142 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:17:03.512142 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:17:03.512142 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:17:03.512142 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:17:03.512142 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:17:03.512142 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:17:03.512142 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:17:03.512142 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:17:03.512142 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:17:03.512142 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:17:03.512142 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:17:03.512142 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:17:03.512142 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:17:03.513850 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:17:03.513850 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:17:03.513850 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:17:03.513850 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:17:03.513850 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:17:03.513850 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:17:03.513850 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:17:03.513850 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:17:03.513850 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:17:03.513850 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:17:03.513850 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:17:03.513850 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:17:03.513850 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:17:03.513850 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:17:03.513850 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:17:03.513850 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:17:03.513850 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:17:03.516247 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:17:03.516247 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:17:03.516247 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:17:03.516247 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:17:03.516247 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:17:03.516247 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:17:03.516247 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:17:03.516247 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:17:03.518368 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 599/1217] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:17:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 391 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:17:03.525317 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a35e149-19c3-49cd-a540-0eee4d2794cb None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:03.790997 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a35e149-19c3-49cd-a540-0eee4d2794cb None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:03.863597 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:03.900956 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e772e84f-c0d0-445f-b684-6d2332edebc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:03.902282 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e772e84f-c0d0-445f-b684-6d2332edebc5 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:03.902811 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e772e84f-c0d0-445f-b684-6d2332edebc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:03.903564 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e772e84f-c0d0-445f-b684-6d2332edebc5 None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:03.905350 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e772e84f-c0d0-445f-b684-6d2332edebc5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:17:03.905975 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e772e84f-c0d0-445f-b684-6d2332edebc5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:03.911073 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e772e84f-c0d0-445f-b684-6d2332edebc5 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:03.945758 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 619/1218] 10.222.0.87 () {56 vars in 1148 bytes} [Tue Jun 21 18:17:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 418 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:17:03.952759 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22537b23-3401-4fe6-b7d1-ec80283c0d8c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:04.220152 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22537b23-3401-4fe6-b7d1-ec80283c0d8c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:04.300349 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:04.318644 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d04c56b-8192-4ad8-b5f5-43598e09e72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:04.319235 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d04c56b-8192-4ad8-b5f5-43598e09e72d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:04.319522 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d04c56b-8192-4ad8-b5f5-43598e09e72d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:04.319865 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d04c56b-8192-4ad8-b5f5-43598e09e72d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:04.321184 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d04c56b-8192-4ad8-b5f5-43598e09e72d None admin] RBAC: Adding query filter params (role.id=b9dbf55e6f2947158c1d946f6ee9f907, scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb, user.id=ef1d49b36cdc4665b82487be97e9dcd5) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:17:04.322104 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d04c56b-8192-4ad8-b5f5-43598e09e72d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:04.323517 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d04c56b-8192-4ad8-b5f5-43598e09e72d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:04.335602 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 600/1219] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:17:03 2022] GET /identity/v3/role_assignments?user.id=ef1d49b36cdc4665b82487be97e9dcd5&scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb&role.id=b9dbf55e6f2947158c1d946f6ee9f907 => generated 259 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:17:07.681469 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20be501-b19c-4592-98a8-cb9ebe8cf378 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:07.682126 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20be501-b19c-4592-98a8-cb9ebe8cf378 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:07.682559 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20be501-b19c-4592-98a8-cb9ebe8cf378 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:07.685342 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 620/1220] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:17:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:17:07.697994 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b388e36-f981-424b-bd55-f9ef6900a506 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:07.699140 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b388e36-f981-424b-bd55-f9ef6900a506 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:07.699140 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b388e36-f981-424b-bd55-f9ef6900a506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:07.726162 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-7b388e36-f981-424b-bd55-f9ef6900a506 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:17:07.729760 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7b388e36-f981-424b-bd55-f9ef6900a506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:17:07.887785 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 601/1221] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:17:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:17:07.938838 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20be501-b19c-4592-98a8-cb9ebe8cf378 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:08.167800 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20be501-b19c-4592-98a8-cb9ebe8cf378 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:08.252531 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f58b281-8203-4174-ae81-b80e7eb21ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:08.253869 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58b281-8203-4174-ae81-b80e7eb21ca6 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:08.254327 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58b281-8203-4174-ae81-b80e7eb21ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:08.254796 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58b281-8203-4174-ae81-b80e7eb21ca6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:08.282048 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f58b281-8203-4174-ae81-b80e7eb21ca6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:17:08.296356 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-5f58b281-8203-4174-ae81-b80e7eb21ca6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:17:08.296356 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:17:08.296356 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:17:08.296356 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:17:08.296356 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:17:08.296356 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:17:08.296356 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:17:08.296356 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:17:08.296356 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:17:08.296356 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:17:08.296356 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:17:08.296356 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:17:08.296356 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:17:08.296356 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:17:08.296356 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:17:08.296356 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:17:08.298943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:17:08.298943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:17:08.298943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:17:08.298943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:17:08.298943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:17:08.298943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:17:08.298943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:17:08.298943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:17:08.298943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:17:08.298943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:17:08.298943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:17:08.298943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:17:08.298943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:17:08.298943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:17:08.298943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:17:08.298943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:17:08.300730 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:17:08.300730 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:17:08.300730 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:17:08.300730 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:17:08.300730 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:17:08.300730 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:17:08.300730 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:17:08.300730 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:17:08.300730 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:17:08.300730 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:17:08.300730 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:17:08.300730 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:17:08.300730 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 621/1222] 10.222.0.87 () {56 vars in 1102 bytes} [Tue Jun 21 18:17:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:17:08.308256 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b388e36-f981-424b-bd55-f9ef6900a506 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:08.506197 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b388e36-f981-424b-bd55-f9ef6900a506 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:08.548246 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:08.569091 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c510007-9453-4426-aaf4-d0d8caf7fb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:08.570189 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c510007-9453-4426-aaf4-d0d8caf7fb4b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:08.570625 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c510007-9453-4426-aaf4-d0d8caf7fb4b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:08.571268 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c510007-9453-4426-aaf4-d0d8caf7fb4b None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:08.572557 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c510007-9453-4426-aaf4-d0d8caf7fb4b None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:17:08.572907 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c510007-9453-4426-aaf4-d0d8caf7fb4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:08.575283 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c510007-9453-4426-aaf4-d0d8caf7fb4b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:08.587830 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 602/1223] 10.222.0.87 () {56 vars in 1103 bytes} [Tue Jun 21 18:17:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:17:08.596479 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f58b281-8203-4174-ae81-b80e7eb21ca6 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:08.672049 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:08.787652 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f58b281-8203-4174-ae81-b80e7eb21ca6 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:08.851163 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8fbe3-5396-40cf-b97c-55e7792d4a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:08.852533 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8fbe3-5396-40cf-b97c-55e7792d4a6d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:08.853031 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8fbe3-5396-40cf-b97c-55e7792d4a6d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:08.853488 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8fbe3-5396-40cf-b97c-55e7792d4a6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:08.873396 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb8fbe3-5396-40cf-b97c-55e7792d4a6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:08.875238 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb8fbe3-5396-40cf-b97c-55e7792d4a6d None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:08.903883 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-5fb8fbe3-5396-40cf-b97c-55e7792d4a6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:17:08.903883 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:17:08.903883 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:17:08.903883 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:17:08.903883 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:17:08.903883 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:17:08.903883 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:17:08.903883 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:17:08.903883 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:17:08.903883 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:17:08.903883 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:17:08.903883 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:17:08.903883 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:17:08.903883 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:17:08.903883 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:17:08.906254 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:17:08.906254 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:17:08.906254 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:17:08.906254 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:17:08.906254 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:17:08.906254 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:17:08.906254 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:17:08.906254 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:17:08.906254 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:17:08.906254 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:17:08.906254 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:17:08.906254 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:17:08.906254 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:17:08.906254 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:17:08.906254 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:17:08.906254 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:17:08.906254 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:17:08.908667 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:17:08.908667 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:17:08.908667 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:17:08.908667 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:17:08.908667 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 622/1224] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:17:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:17:08.913918 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c510007-9453-4426-aaf4-d0d8caf7fb4b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:09.138414 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c510007-9453-4426-aaf4-d0d8caf7fb4b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:09.196629 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7d1fa9-c2a3-4dbb-af81-958a9efaf02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:09.197887 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d1fa9-c2a3-4dbb-af81-958a9efaf02d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:09.198208 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d1fa9-c2a3-4dbb-af81-958a9efaf02d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:09.198626 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d1fa9-c2a3-4dbb-af81-958a9efaf02d None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:09.199718 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7d1fa9-c2a3-4dbb-af81-958a9efaf02d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:17:09.200036 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7d1fa9-c2a3-4dbb-af81-958a9efaf02d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:09.201879 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7d1fa9-c2a3-4dbb-af81-958a9efaf02d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:09.220757 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 603/1225] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:17:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:17:09.227806 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8fbe3-5396-40cf-b97c-55e7792d4a6d None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:09.445689 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:09.463944 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8fbe3-5396-40cf-b97c-55e7792d4a6d None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:09.527197 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d592733-178b-42f0-9fd3-b095aa546cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:09.528280 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d592733-178b-42f0-9fd3-b095aa546cc5 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:09.528562 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d592733-178b-42f0-9fd3-b095aa546cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:09.528835 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d592733-178b-42f0-9fd3-b095aa546cc5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:09.542082 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d592733-178b-42f0-9fd3-b095aa546cc5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:09.543886 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d592733-178b-42f0-9fd3-b095aa546cc5 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:09.563517 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-7d592733-178b-42f0-9fd3-b095aa546cc5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 21 18:17:09.563517 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:17:09.563517 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:17:09.563517 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:17:09.563517 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:17:09.563517 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:17:09.563517 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:17:09.563517 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:17:09.563517 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:17:09.563517 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:17:09.563517 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:17:09.563517 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:17:09.563517 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:17:09.563517 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:17:09.563517 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:17:09.563517 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:17:09.565913 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:17:09.565913 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:17:09.565913 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:17:09.565913 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:17:09.565913 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:17:09.565913 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:17:09.565913 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:17:09.565913 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:17:09.565913 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:17:09.565913 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:17:09.565913 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:17:09.565913 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:17:09.565913 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:17:09.565913 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:17:09.565913 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:17:09.565913 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:17:09.565913 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:17:09.565913 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:17:09.568027 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:17:09.568027 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:17:09.568027 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:17:09.568027 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:17:09.568027 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:17:09.568027 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:17:09.568027 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:17:09.568027 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:17:09.568027 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 21 18:17:09.568027 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:17:09.568027 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 623/1226] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:17:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:17:09.575522 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7d1fa9-c2a3-4dbb-af81-958a9efaf02d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:09.671558 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:09.760232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7d1fa9-c2a3-4dbb-af81-958a9efaf02d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:09.812086 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ce44e-f550-436c-969e-7856024eb262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:09.813318 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ce44e-f550-436c-969e-7856024eb262 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:09.814050 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ce44e-f550-436c-969e-7856024eb262 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:09.814528 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ce44e-f550-436c-969e-7856024eb262 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:09.816079 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119ce44e-f550-436c-969e-7856024eb262 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:17:09.817487 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119ce44e-f550-436c-969e-7856024eb262 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:09.818725 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119ce44e-f550-436c-969e-7856024eb262 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:09.845341 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 604/1227] 10.222.0.87 () {56 vars in 1143 bytes} [Tue Jun 21 18:17:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:17:09.852935 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d592733-178b-42f0-9fd3-b095aa546cc5 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:09.993138 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d592733-178b-42f0-9fd3-b095aa546cc5 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:10.057271 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ebb76-a918-44e2-b922-a8c6770f17a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:10.058833 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ebb76-a918-44e2-b922-a8c6770f17a3 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:10.059431 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ebb76-a918-44e2-b922-a8c6770f17a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:10.060013 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ebb76-a918-44e2-b922-a8c6770f17a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:10.078651 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758ebb76-a918-44e2-b922-a8c6770f17a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:10.080765 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758ebb76-a918-44e2-b922-a8c6770f17a3 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:10.107244 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-758ebb76-a918-44e2-b922-a8c6770f17a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:17:10.107244 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:17:10.107244 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:17:10.107244 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:17:10.107244 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:17:10.107244 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:17:10.107244 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:17:10.107244 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:17:10.107244 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:17:10.107244 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:17:10.107244 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:17:10.107244 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:17:10.107244 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:17:10.107244 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:17:10.107244 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:17:10.109520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:17:10.109520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:17:10.109520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:17:10.109520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:17:10.109520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:17:10.109520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:17:10.109520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:17:10.109520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:17:10.109520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:17:10.109520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:17:10.109520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:17:10.109520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:17:10.109520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:17:10.109520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:17:10.109520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:17:10.109520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:17:10.109520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:17:10.111468 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:17:10.111468 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:17:10.111468 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:17:10.111468 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:17:10.111468 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 624/1228] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:17:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:17:10.117405 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ce44e-f550-436c-969e-7856024eb262 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:10.319541 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ce44e-f550-436c-969e-7856024eb262 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:10.324095 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:10.393991 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7765a2c6-496c-4e84-bbbd-44d5235c17ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:10.394795 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765a2c6-496c-4e84-bbbd-44d5235c17ec None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:10.395152 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765a2c6-496c-4e84-bbbd-44d5235c17ec None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:10.395516 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765a2c6-496c-4e84-bbbd-44d5235c17ec None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:10.396438 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7765a2c6-496c-4e84-bbbd-44d5235c17ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:17:10.396905 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7765a2c6-496c-4e84-bbbd-44d5235c17ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:10.398698 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7765a2c6-496c-4e84-bbbd-44d5235c17ec None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:10.414526 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 605/1229] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:17:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:17:10.425271 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ebb76-a918-44e2-b922-a8c6770f17a3 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:10.441932 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:10.607792 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ebb76-a918-44e2-b922-a8c6770f17a3 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:10.671400 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8c9b78-5b0a-4f93-85ac-9acf142a59fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:10.672344 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c9b78-5b0a-4f93-85ac-9acf142a59fc None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:10.672816 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c9b78-5b0a-4f93-85ac-9acf142a59fc None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:10.673156 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c9b78-5b0a-4f93-85ac-9acf142a59fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:10.686200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8c9b78-5b0a-4f93-85ac-9acf142a59fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:10.687852 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8c9b78-5b0a-4f93-85ac-9acf142a59fc None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:10.713369 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-db8c9b78-5b0a-4f93-85ac-9acf142a59fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:17:10.713369 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:17:10.713369 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:17:10.713369 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:17:10.713369 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:17:10.713369 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:17:10.713369 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:17:10.713369 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:17:10.713369 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:17:10.713369 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:17:10.713369 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:17:10.713369 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:17:10.713369 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:17:10.713369 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:17:10.713369 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:17:10.714521 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:17:10.714521 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:17:10.714521 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:17:10.714521 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:17:10.714521 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:17:10.714521 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:17:10.714521 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:17:10.714521 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:17:10.714521 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:17:10.714521 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:17:10.714521 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:17:10.714521 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:17:10.714521 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:17:10.714521 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:17:10.714521 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:17:10.714521 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:17:10.714521 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:17:10.715584 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:17:10.715584 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:17:10.715584 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:17:10.715584 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:17:10.715584 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:17:10.715584 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:17:10.715584 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:17:10.715584 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:17:10.715584 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 625/1230] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:17:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:17:10.722883 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7765a2c6-496c-4e84-bbbd-44d5235c17ec None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:10.907475 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7765a2c6-496c-4e84-bbbd-44d5235c17ec None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:10.964553 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac177a5a-f93f-4861-83fa-54a25b3123dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:10.965270 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac177a5a-f93f-4861-83fa-54a25b3123dc None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:10.965629 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac177a5a-f93f-4861-83fa-54a25b3123dc None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:10.966025 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac177a5a-f93f-4861-83fa-54a25b3123dc None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:10.967092 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac177a5a-f93f-4861-83fa-54a25b3123dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:17:10.967605 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac177a5a-f93f-4861-83fa-54a25b3123dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:10.969373 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac177a5a-f93f-4861-83fa-54a25b3123dc None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:10.991308 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 606/1231] 10.222.0.87 () {56 vars in 1148 bytes} [Tue Jun 21 18:17:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:17:10.999616 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8c9b78-5b0a-4f93-85ac-9acf142a59fc None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:11.107327 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:11.185678 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8c9b78-5b0a-4f93-85ac-9acf142a59fc None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:11.242879 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c523080-4df0-4e51-8bd3-fdbf27168e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:11.244608 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c523080-4df0-4e51-8bd3-fdbf27168e57 None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:11.245093 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c523080-4df0-4e51-8bd3-fdbf27168e57 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:11.245584 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c523080-4df0-4e51-8bd3-fdbf27168e57 None admin] PATH_INFO: `/v3/projects/5f30c3ddb5e44e9299ff228fe94c27fb/users/ef1d49b36cdc4665b82487be97e9dcd5/roles/b9dbf55e6f2947158c1d946f6ee9f907` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:11.275961 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c523080-4df0-4e51-8bd3-fdbf27168e57 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f30c3ddb5e44e9299ff228fe94c27fb, user_id=ef1d49b36cdc4665b82487be97e9dcd5, role_id=b9dbf55e6f2947158c1d946f6ee9f907)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:11.278877 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c523080-4df0-4e51-8bd3-fdbf27168e57 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:11.305172 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 626/1232] 10.222.0.87 () {58 vars in 1439 bytes} [Tue Jun 21 18:17:10 2022] PUT /identity/v3/projects/5f30c3ddb5e44e9299ff228fe94c27fb/users/ef1d49b36cdc4665b82487be97e9dcd5/roles/b9dbf55e6f2947158c1d946f6ee9f907 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:17:14.059808 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9b2b3-4161-4240-817e-f5565a26636f None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:14.060712 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9b2b3-4161-4240-817e-f5565a26636f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:14.060712 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9b2b3-4161-4240-817e-f5565a26636f None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:14.062561 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 607/1233] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:17:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:17:14.070352 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b86984-2555-402f-9591-9e8ad2853315 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:14.070907 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b86984-2555-402f-9591-9e8ad2853315 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:14.070907 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b86984-2555-402f-9591-9e8ad2853315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:14.090387 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-35b86984-2555-402f-9591-9e8ad2853315 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:17:14.091858 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-35b86984-2555-402f-9591-9e8ad2853315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:17:14.192797 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 627/1234] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:17:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:17:14.217877 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d9b2b3-4161-4240-817e-f5565a26636f None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:14.249618 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:14.369881 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d9b2b3-4161-4240-817e-f5565a26636f None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:14.421883 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ad003b-cbf4-403e-9bb8-61602f538227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:14.422794 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad003b-cbf4-403e-9bb8-61602f538227 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:14.423045 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad003b-cbf4-403e-9bb8-61602f538227 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:14.423362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad003b-cbf4-403e-9bb8-61602f538227 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:14.439675 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ad003b-cbf4-403e-9bb8-61602f538227 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:17:14.448435 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-b2ad003b-cbf4-403e-9bb8-61602f538227 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:17:14.448435 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:17:14.448435 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:17:14.448435 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:17:14.448435 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:17:14.448435 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:17:14.448435 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:17:14.448435 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:17:14.448435 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:17:14.448435 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:17:14.448435 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:17:14.448435 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:17:14.448435 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:17:14.448435 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:17:14.448435 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:17:14.448435 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:17:14.450431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:17:14.450431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:17:14.450431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:17:14.450431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:17:14.450431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:17:14.450431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:17:14.450431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:17:14.450431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:17:14.450431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:17:14.450431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:17:14.450431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:17:14.450431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:17:14.450431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:17:14.450431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:17:14.450431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:17:14.450431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:17:14.452026 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:17:14.452026 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:17:14.452026 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:17:14.452026 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:17:14.452026 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:17:14.452026 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:17:14.452026 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:17:14.452026 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:17:14.452026 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:17:14.452026 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:17:14.452026 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:17:14.452026 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:17:14.452026 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 608/1235] 10.222.0.87 () {56 vars in 1102 bytes} [Tue Jun 21 18:17:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:17:14.459677 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b86984-2555-402f-9591-9e8ad2853315 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:14.533200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:14.621474 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b86984-2555-402f-9591-9e8ad2853315 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:14.677216 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91897b7c-3e4f-4ef9-9080-47e84da7eca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:14.678006 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91897b7c-3e4f-4ef9-9080-47e84da7eca3 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:14.678296 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91897b7c-3e4f-4ef9-9080-47e84da7eca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:14.678577 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91897b7c-3e4f-4ef9-9080-47e84da7eca3 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:14.679363 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91897b7c-3e4f-4ef9-9080-47e84da7eca3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:17:14.679680 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91897b7c-3e4f-4ef9-9080-47e84da7eca3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:14.681139 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91897b7c-3e4f-4ef9-9080-47e84da7eca3 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:14.701965 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 628/1236] 10.222.0.87 () {56 vars in 1103 bytes} [Tue Jun 21 18:17:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:17:14.704299 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ad003b-cbf4-403e-9bb8-61602f538227 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:14.849465 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:14.870839 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ad003b-cbf4-403e-9bb8-61602f538227 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:14.919486 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6202d70-e50f-483a-bf75-31be0e081678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:14.920445 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6202d70-e50f-483a-bf75-31be0e081678 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:14.920827 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6202d70-e50f-483a-bf75-31be0e081678 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:14.921219 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6202d70-e50f-483a-bf75-31be0e081678 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:14.934680 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6202d70-e50f-483a-bf75-31be0e081678 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:14.936370 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6202d70-e50f-483a-bf75-31be0e081678 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:14.959492 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-d6202d70-e50f-483a-bf75-31be0e081678 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:17:14.959492 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:17:14.959492 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:17:14.959492 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:17:14.959492 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:17:14.959492 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:17:14.959492 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:17:14.959492 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:17:14.959492 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:17:14.959492 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:17:14.959492 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:17:14.959492 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:17:14.959492 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:17:14.959492 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:17:14.959492 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:17:14.961500 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:17:14.961500 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:17:14.961500 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:17:14.961500 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:17:14.961500 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:17:14.961500 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:17:14.961500 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:17:14.961500 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:17:14.961500 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:17:14.961500 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:17:14.961500 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:17:14.961500 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:17:14.961500 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:17:14.961500 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:17:14.961500 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:17:14.961500 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:17:14.961500 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:17:14.963699 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:17:14.963699 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:17:14.963699 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:17:14.963699 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:17:14.963699 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 609/1237] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:17:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:17:14.969173 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91897b7c-3e4f-4ef9-9080-47e84da7eca3 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:15.135730 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91897b7c-3e4f-4ef9-9080-47e84da7eca3 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:15.150130 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:15.190563 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72777354-5cb8-4a57-8e7e-331aee7386f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:15.191424 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72777354-5cb8-4a57-8e7e-331aee7386f4 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:15.191811 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72777354-5cb8-4a57-8e7e-331aee7386f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:15.192206 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72777354-5cb8-4a57-8e7e-331aee7386f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:15.193317 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72777354-5cb8-4a57-8e7e-331aee7386f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:17:15.193797 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72777354-5cb8-4a57-8e7e-331aee7386f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:15.195389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72777354-5cb8-4a57-8e7e-331aee7386f4 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:15.215531 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 629/1238] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:17:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:17:15.225728 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6202d70-e50f-483a-bf75-31be0e081678 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:15.412794 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6202d70-e50f-483a-bf75-31be0e081678 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:15.474033 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f990b0b3-eaea-448d-a6b5-a25c27ee447d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:15.475365 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f990b0b3-eaea-448d-a6b5-a25c27ee447d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:15.475898 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f990b0b3-eaea-448d-a6b5-a25c27ee447d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:15.476480 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f990b0b3-eaea-448d-a6b5-a25c27ee447d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:15.483023 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:15.497542 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f990b0b3-eaea-448d-a6b5-a25c27ee447d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:15.501225 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f990b0b3-eaea-448d-a6b5-a25c27ee447d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:15.525554 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-f990b0b3-eaea-448d-a6b5-a25c27ee447d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 21 18:17:15.525554 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:17:15.525554 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:17:15.525554 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:17:15.525554 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:17:15.525554 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:17:15.525554 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:17:15.525554 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:17:15.525554 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:17:15.525554 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:17:15.525554 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:17:15.525554 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:17:15.525554 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:17:15.525554 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:17:15.525554 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:17:15.525554 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:17:15.527944 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:17:15.527944 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:17:15.527944 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:17:15.527944 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:17:15.527944 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:17:15.527944 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:17:15.527944 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:17:15.527944 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:17:15.527944 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:17:15.527944 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:17:15.527944 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:17:15.527944 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:17:15.527944 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:17:15.527944 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:17:15.527944 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:17:15.527944 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:17:15.527944 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:17:15.527944 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:17:15.530151 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:17:15.530151 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:17:15.530151 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:17:15.530151 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:17:15.530151 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:17:15.530151 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:17:15.530151 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:17:15.530151 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:17:15.530151 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 21 18:17:15.530151 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:17:15.530151 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 610/1239] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:17:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:17:15.535007 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72777354-5cb8-4a57-8e7e-331aee7386f4 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:15.683911 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72777354-5cb8-4a57-8e7e-331aee7386f4 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:15.730654 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75bf339-ee69-473f-9dda-cd5d039d2149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:15.731930 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75bf339-ee69-473f-9dda-cd5d039d2149 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:15.732343 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75bf339-ee69-473f-9dda-cd5d039d2149 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:15.732634 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75bf339-ee69-473f-9dda-cd5d039d2149 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:15.733726 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75bf339-ee69-473f-9dda-cd5d039d2149 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:17:15.734242 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75bf339-ee69-473f-9dda-cd5d039d2149 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:15.736918 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75bf339-ee69-473f-9dda-cd5d039d2149 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:15.761506 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 630/1240] 10.222.0.87 () {56 vars in 1143 bytes} [Tue Jun 21 18:17:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:17:15.769607 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f990b0b3-eaea-448d-a6b5-a25c27ee447d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:15.977282 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f990b0b3-eaea-448d-a6b5-a25c27ee447d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:16.041502 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b44eae3-02bb-4f7a-a562-80d06112beda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:16.042735 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b44eae3-02bb-4f7a-a562-80d06112beda None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:16.043238 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b44eae3-02bb-4f7a-a562-80d06112beda None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:16.043935 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b44eae3-02bb-4f7a-a562-80d06112beda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:16.187892 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:16.197162 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b44eae3-02bb-4f7a-a562-80d06112beda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:16.198792 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b44eae3-02bb-4f7a-a562-80d06112beda None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:16.343193 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 611/1241] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:17:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 575 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:17:16.352770 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75bf339-ee69-473f-9dda-cd5d039d2149 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:16.394119 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:16.502962 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75bf339-ee69-473f-9dda-cd5d039d2149 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:16.566018 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f252054-1d6c-44b4-b170-0d0e2b6a3452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:16.567320 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f252054-1d6c-44b4-b170-0d0e2b6a3452 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:16.567952 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f252054-1d6c-44b4-b170-0d0e2b6a3452 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:16.568375 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f252054-1d6c-44b4-b170-0d0e2b6a3452 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:16.588647 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f252054-1d6c-44b4-b170-0d0e2b6a3452 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:16.590472 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f252054-1d6c-44b4-b170-0d0e2b6a3452 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:16.628627 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-6f252054-1d6c-44b4-b170-0d0e2b6a3452 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:17:16.628627 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:17:16.628627 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:17:16.628627 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:17:16.628627 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:17:16.628627 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:17:16.628627 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:17:16.628627 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:17:16.628627 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:17:16.628627 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:17:16.628627 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:17:16.628627 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:17:16.628627 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:17:16.628627 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:17:16.628627 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:17:16.630564 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:17:16.630564 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:17:16.630564 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:17:16.630564 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:17:16.630564 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:17:16.630564 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:17:16.630564 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:17:16.630564 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:17:16.630564 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:17:16.630564 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:17:16.630564 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:17:16.630564 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:17:16.630564 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:17:16.630564 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:17:16.630564 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:17:16.630564 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:17:16.630564 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:17:16.632335 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:17:16.632335 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:17:16.632335 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:17:16.632335 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:17:16.633086 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 631/1242] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:17:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:17:16.641572 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b44eae3-02bb-4f7a-a562-80d06112beda None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:16.811077 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:16.820232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b44eae3-02bb-4f7a-a562-80d06112beda None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:16.896491 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbcb460-4406-4f8d-ae93-c267678740bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:16.899332 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbcb460-4406-4f8d-ae93-c267678740bb None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:16.900367 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbcb460-4406-4f8d-ae93-c267678740bb None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:16.900367 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbcb460-4406-4f8d-ae93-c267678740bb None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:16.903314 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbcb460-4406-4f8d-ae93-c267678740bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:17:16.903660 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbcb460-4406-4f8d-ae93-c267678740bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:16.906954 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbcb460-4406-4f8d-ae93-c267678740bb None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:16.948763 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 612/1243] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:17:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:17:16.951027 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f252054-1d6c-44b4-b170-0d0e2b6a3452 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:17.077808 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:17.112785 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f252054-1d6c-44b4-b170-0d0e2b6a3452 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:17.168589 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c89dd2-ef95-4841-b4f9-f184376d51da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:17.169484 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c89dd2-ef95-4841-b4f9-f184376d51da None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:17.169903 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c89dd2-ef95-4841-b4f9-f184376d51da None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:17.170299 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c89dd2-ef95-4841-b4f9-f184376d51da None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:17.181287 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c89dd2-ef95-4841-b4f9-f184376d51da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:17.183701 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c89dd2-ef95-4841-b4f9-f184376d51da None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:17.200949 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-21c89dd2-ef95-4841-b4f9-f184376d51da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:17:17.200949 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:17:17.200949 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:17:17.200949 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:17:17.200949 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:17:17.200949 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:17:17.200949 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:17:17.200949 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:17:17.200949 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:17:17.200949 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:17:17.200949 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:17:17.200949 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:17:17.200949 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:17:17.200949 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:17:17.200949 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:17:17.202935 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:17:17.202935 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:17:17.202935 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:17:17.202935 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:17:17.202935 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:17:17.202935 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:17:17.202935 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:17:17.202935 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:17:17.202935 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:17:17.202935 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:17:17.202935 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:17:17.202935 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:17:17.202935 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:17:17.202935 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:17:17.202935 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:17:17.202935 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:17:17.202935 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:17:17.204710 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:17:17.204710 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:17:17.204710 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:17:17.204710 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:17:17.204710 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:17:17.204710 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:17:17.204710 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:17:17.204710 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:17:17.204710 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 632/1244] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:17:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:17:17.209536 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbcb460-4406-4f8d-ae93-c267678740bb None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:17.383462 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbcb460-4406-4f8d-ae93-c267678740bb None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:17.459831 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0162ec83-4acb-4945-831f-a25f384e50c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:17.460991 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162ec83-4acb-4945-831f-a25f384e50c0 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:17.461491 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162ec83-4acb-4945-831f-a25f384e50c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:17.462232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162ec83-4acb-4945-831f-a25f384e50c0 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:17.463972 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0162ec83-4acb-4945-831f-a25f384e50c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:17:17.464956 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0162ec83-4acb-4945-831f-a25f384e50c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:17.467438 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0162ec83-4acb-4945-831f-a25f384e50c0 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:17.495847 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 613/1245] 10.222.0.87 () {56 vars in 1148 bytes} [Tue Jun 21 18:17:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:17:17.507006 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c89dd2-ef95-4841-b4f9-f184376d51da None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:17.701706 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c89dd2-ef95-4841-b4f9-f184376d51da None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:17.773155 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0222833d-3523-4402-96d5-cf414f6c4e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:17.774360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222833d-3523-4402-96d5-cf414f6c4e96 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:17.774887 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222833d-3523-4402-96d5-cf414f6c4e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:17.776440 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222833d-3523-4402-96d5-cf414f6c4e96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:17.778152 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0222833d-3523-4402-96d5-cf414f6c4e96 None admin] RBAC: Adding query filter params (role.id=b9dbf55e6f2947158c1d946f6ee9f907, scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb, user.id=ef1d49b36cdc4665b82487be97e9dcd5) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:17:17.778704 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0222833d-3523-4402-96d5-cf414f6c4e96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:17.781445 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0222833d-3523-4402-96d5-cf414f6c4e96 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:17.796272 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 633/1246] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:17:17 2022] GET /identity/v3/role_assignments?user.id=ef1d49b36cdc4665b82487be97e9dcd5&scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb&role.id=b9dbf55e6f2947158c1d946f6ee9f907 => generated 608 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:17:20.443278 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b839c8-d97c-4d5a-9c44-a0cd7cab93c3 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:20.444036 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b839c8-d97c-4d5a-9c44-a0cd7cab93c3 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:20.444036 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b839c8-d97c-4d5a-9c44-a0cd7cab93c3 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:20.445560 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 614/1247] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:17:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:17:20.453371 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc01857-5885-4d97-bb0d-d1625fcd8f32 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:20.454065 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc01857-5885-4d97-bb0d-d1625fcd8f32 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:20.454311 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc01857-5885-4d97-bb0d-d1625fcd8f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:20.463331 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:20.484642 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-6dc01857-5885-4d97-bb0d-d1625fcd8f32 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:17:20.486154 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-6dc01857-5885-4d97-bb0d-d1625fcd8f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:17:20.612867 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 634/1248] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:17:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:17:20.656349 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b839c8-d97c-4d5a-9c44-a0cd7cab93c3 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:20.691007 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:20.838155 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b839c8-d97c-4d5a-9c44-a0cd7cab93c3 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:20.900764 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4e5826-c291-42c9-94d1-91724e4cdbaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:20.902043 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e5826-c291-42c9-94d1-91724e4cdbaf None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:20.902530 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e5826-c291-42c9-94d1-91724e4cdbaf None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:20.903058 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e5826-c291-42c9-94d1-91724e4cdbaf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:20.920795 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4e5826-c291-42c9-94d1-91724e4cdbaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:17:20.933442 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-ee4e5826-c291-42c9-94d1-91724e4cdbaf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:17:20.933442 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:17:20.933442 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:17:20.933442 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:17:20.933442 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:17:20.933442 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:17:20.933442 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:17:20.933442 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:17:20.933442 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:17:20.933442 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:17:20.933442 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:17:20.933442 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:17:20.933442 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:17:20.933442 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:17:20.933442 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:17:20.933442 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:17:20.935148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:17:20.935148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:17:20.935148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:17:20.935148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:17:20.935148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:17:20.935148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:17:20.935148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:17:20.935148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:17:20.935148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:17:20.935148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:17:20.935148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:17:20.935148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:17:20.935148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:17:20.935148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:17:20.935148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:17:20.935148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:17:20.936129 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:17:20.936129 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:17:20.936129 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:17:20.936129 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:17:20.936129 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:17:20.936129 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:17:20.936129 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:17:20.936129 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:17:20.936129 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:17:20.936129 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:17:20.936129 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:17:20.936129 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:17:20.936129 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 615/1249] 10.222.0.87 () {56 vars in 1096 bytes} [Tue Jun 21 18:17:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:17:20.947678 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc01857-5885-4d97-bb0d-d1625fcd8f32 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:21.137761 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc01857-5885-4d97-bb0d-d1625fcd8f32 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:21.196092 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:21.202925 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7755bad-a375-43e1-9aae-c1d8883c1d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:21.203604 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7755bad-a375-43e1-9aae-c1d8883c1d35 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:21.203889 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7755bad-a375-43e1-9aae-c1d8883c1d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:21.204223 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7755bad-a375-43e1-9aae-c1d8883c1d35 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:21.204995 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7755bad-a375-43e1-9aae-c1d8883c1d35 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:17:21.205303 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7755bad-a375-43e1-9aae-c1d8883c1d35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:21.212185 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7755bad-a375-43e1-9aae-c1d8883c1d35 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:21.218998 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 635/1250] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:17:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:17:21.228994 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4e5826-c291-42c9-94d1-91724e4cdbaf None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:21.367065 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:21.418280 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4e5826-c291-42c9-94d1-91724e4cdbaf None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:21.482823 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc2245-b576-427f-b94a-5d3f6c5144a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:21.484381 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc2245-b576-427f-b94a-5d3f6c5144a9 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:21.484989 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc2245-b576-427f-b94a-5d3f6c5144a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:21.485575 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc2245-b576-427f-b94a-5d3f6c5144a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:21.514697 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc2245-b576-427f-b94a-5d3f6c5144a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:21.517309 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc2245-b576-427f-b94a-5d3f6c5144a9 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:21.554205 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-5bdc2245-b576-427f-b94a-5d3f6c5144a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:17:21.554205 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:17:21.554205 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:17:21.554205 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:17:21.554205 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:17:21.554205 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:17:21.554205 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:17:21.554205 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:17:21.554205 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:17:21.554205 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:17:21.554205 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:17:21.554205 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:17:21.554205 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:17:21.554205 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:17:21.554205 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:17:21.556578 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:17:21.556578 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:17:21.556578 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:17:21.556578 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:17:21.556578 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:17:21.556578 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:17:21.556578 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:17:21.556578 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:17:21.556578 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:17:21.556578 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:17:21.556578 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:17:21.556578 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:17:21.556578 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:17:21.556578 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:17:21.556578 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:17:21.556578 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:17:21.556578 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:17:21.558295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:17:21.558295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:17:21.558295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:17:21.558295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:17:21.558295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 616/1251] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:17:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:17:21.565488 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7755bad-a375-43e1-9aae-c1d8883c1d35 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:21.734875 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7755bad-a375-43e1-9aae-c1d8883c1d35 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:21.785463 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4d45a-1e1e-4502-ad44-ecbb6564d632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:21.786456 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4d45a-1e1e-4502-ad44-ecbb6564d632 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:21.786773 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4d45a-1e1e-4502-ad44-ecbb6564d632 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:21.787228 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4d45a-1e1e-4502-ad44-ecbb6564d632 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:21.788234 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae4d45a-1e1e-4502-ad44-ecbb6564d632 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:17:21.788573 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae4d45a-1e1e-4502-ad44-ecbb6564d632 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:21.790340 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae4d45a-1e1e-4502-ad44-ecbb6564d632 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:21.805526 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 636/1252] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:17:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:17:21.813911 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc2245-b576-427f-b94a-5d3f6c5144a9 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:22.001183 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc2245-b576-427f-b94a-5d3f6c5144a9 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:22.048614 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:22.065770 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1582b4e-5c0e-446a-ac9f-d6c8485b8e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:22.067216 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1582b4e-5c0e-446a-ac9f-d6c8485b8e32 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:22.067668 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1582b4e-5c0e-446a-ac9f-d6c8485b8e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:22.068213 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1582b4e-5c0e-446a-ac9f-d6c8485b8e32 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:22.086755 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1582b4e-5c0e-446a-ac9f-d6c8485b8e32 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:22.092162 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1582b4e-5c0e-446a-ac9f-d6c8485b8e32 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:22.125253 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-c1582b4e-5c0e-446a-ac9f-d6c8485b8e32 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 21 18:17:22.125253 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:17:22.125253 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:17:22.125253 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:17:22.125253 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:17:22.125253 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:17:22.125253 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:17:22.125253 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:17:22.125253 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:17:22.125253 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:17:22.125253 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:17:22.125253 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:17:22.125253 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:17:22.125253 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:17:22.125253 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:17:22.125253 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:17:22.126394 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:17:22.126394 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:17:22.126394 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:17:22.126394 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:17:22.126394 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:17:22.126394 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:17:22.126394 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:17:22.126394 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:17:22.126394 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:17:22.126394 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:17:22.126394 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:17:22.126394 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:17:22.126394 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:17:22.126394 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:17:22.126394 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:17:22.126394 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:17:22.126394 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:17:22.126394 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:17:22.127461 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:17:22.127461 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:17:22.127461 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:17:22.127461 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:17:22.127461 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:17:22.127461 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:17:22.127461 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:17:22.127461 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:17:22.127461 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 21 18:17:22.127461 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:17:22.128099 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 617/1253] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:17:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:17:22.138489 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4d45a-1e1e-4502-ad44-ecbb6564d632 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:22.268182 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:22.360057 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4d45a-1e1e-4502-ad44-ecbb6564d632 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:22.424447 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf11062-64cd-4156-bcbe-6ff3319deed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:22.425584 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf11062-64cd-4156-bcbe-6ff3319deed3 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:22.426092 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf11062-64cd-4156-bcbe-6ff3319deed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:22.426394 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf11062-64cd-4156-bcbe-6ff3319deed3 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:22.427418 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf11062-64cd-4156-bcbe-6ff3319deed3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:17:22.427760 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf11062-64cd-4156-bcbe-6ff3319deed3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:22.429219 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf11062-64cd-4156-bcbe-6ff3319deed3 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:22.453832 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 637/1254] 10.222.0.87 () {56 vars in 1143 bytes} [Tue Jun 21 18:17:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:17:22.460796 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1582b4e-5c0e-446a-ac9f-d6c8485b8e32 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:22.628588 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1582b4e-5c0e-446a-ac9f-d6c8485b8e32 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:22.700537 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0694e-b307-489f-9171-65489a5e4bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:22.701622 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0694e-b307-489f-9171-65489a5e4bb9 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:22.701622 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0694e-b307-489f-9171-65489a5e4bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:22.702062 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0694e-b307-489f-9171-65489a5e4bb9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:22.817252 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:22.847057 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a0694e-b307-489f-9171-65489a5e4bb9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:22.849277 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a0694e-b307-489f-9171-65489a5e4bb9 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:23.040629 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 618/1255] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:17:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 580 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:17:23.049871 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf11062-64cd-4156-bcbe-6ff3319deed3 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:23.234792 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf11062-64cd-4156-bcbe-6ff3319deed3 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:23.256371 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:23.292456 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6797ec54-ce83-4157-83b0-096d38f2d991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:23.293451 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6797ec54-ce83-4157-83b0-096d38f2d991 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:23.293891 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6797ec54-ce83-4157-83b0-096d38f2d991 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:23.294292 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6797ec54-ce83-4157-83b0-096d38f2d991 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:23.314390 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6797ec54-ce83-4157-83b0-096d38f2d991 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:23.317062 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6797ec54-ce83-4157-83b0-096d38f2d991 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:23.347758 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-6797ec54-ce83-4157-83b0-096d38f2d991 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:17:23.347758 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:17:23.347758 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:17:23.347758 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:17:23.347758 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:17:23.347758 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:17:23.347758 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:17:23.347758 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:17:23.347758 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:17:23.347758 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:17:23.347758 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:17:23.347758 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:17:23.347758 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:17:23.347758 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:17:23.347758 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:17:23.348983 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:17:23.348983 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:17:23.348983 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:17:23.348983 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:17:23.348983 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:17:23.348983 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:17:23.348983 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:17:23.348983 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:17:23.348983 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:17:23.348983 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:17:23.348983 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:17:23.348983 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:17:23.348983 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:17:23.348983 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:17:23.348983 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:17:23.348983 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:17:23.348983 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:17:23.351182 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:17:23.351182 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:17:23.351182 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:17:23.351182 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:17:23.351182 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 638/1256] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:17:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:17:23.356787 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0694e-b307-489f-9171-65489a5e4bb9 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:23.530640 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:23.548800 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0694e-b307-489f-9171-65489a5e4bb9 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:23.604042 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e21d3f2-ea60-4ec4-9d1d-25fc5f35fcca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:23.604937 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21d3f2-ea60-4ec4-9d1d-25fc5f35fcca None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:23.605304 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21d3f2-ea60-4ec4-9d1d-25fc5f35fcca None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:23.605682 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21d3f2-ea60-4ec4-9d1d-25fc5f35fcca None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:23.606811 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e21d3f2-ea60-4ec4-9d1d-25fc5f35fcca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:17:23.607116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e21d3f2-ea60-4ec4-9d1d-25fc5f35fcca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:23.608688 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e21d3f2-ea60-4ec4-9d1d-25fc5f35fcca None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:23.633434 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 619/1257] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:17:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:17:23.641662 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6797ec54-ce83-4157-83b0-096d38f2d991 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:23.843870 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6797ec54-ce83-4157-83b0-096d38f2d991 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:23.915388 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f139fc-0bdf-49ad-b9d7-dfbd90ed6d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:23.917148 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f139fc-0bdf-49ad-b9d7-dfbd90ed6d54 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:23.917671 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f139fc-0bdf-49ad-b9d7-dfbd90ed6d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:23.918162 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f139fc-0bdf-49ad-b9d7-dfbd90ed6d54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:23.934049 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f139fc-0bdf-49ad-b9d7-dfbd90ed6d54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:23.937130 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f139fc-0bdf-49ad-b9d7-dfbd90ed6d54 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:23.968812 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-a9f139fc-0bdf-49ad-b9d7-dfbd90ed6d54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:17:23.968812 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:17:23.968812 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:17:23.968812 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:17:23.968812 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:17:23.968812 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:17:23.968812 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:17:23.968812 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:17:23.968812 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:17:23.968812 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:17:23.968812 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:17:23.968812 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:17:23.968812 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:17:23.968812 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:17:23.968812 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:17:23.970065 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:17:23.970065 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:17:23.970065 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:17:23.970065 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:17:23.970065 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:17:23.970065 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:17:23.970065 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:17:23.970065 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:17:23.970065 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:17:23.970065 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:17:23.970065 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:17:23.970065 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:17:23.970065 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:17:23.970065 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:17:23.970065 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:17:23.970065 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:17:23.970065 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:17:23.971171 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:17:23.971171 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:17:23.971171 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:17:23.971171 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:17:23.971171 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:17:23.971171 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:17:23.971171 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:17:23.971171 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:17:23.978169 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 639/1258] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:17:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:17:23.981920 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e21d3f2-ea60-4ec4-9d1d-25fc5f35fcca None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:24.158983 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e21d3f2-ea60-4ec4-9d1d-25fc5f35fcca None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:24.219013 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720332c9-d052-4ec6-83cb-39ce955bc940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:24.220133 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720332c9-d052-4ec6-83cb-39ce955bc940 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:24.220634 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720332c9-d052-4ec6-83cb-39ce955bc940 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:24.221001 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720332c9-d052-4ec6-83cb-39ce955bc940 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:24.222245 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720332c9-d052-4ec6-83cb-39ce955bc940 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:17:24.222650 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720332c9-d052-4ec6-83cb-39ce955bc940 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:24.224496 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720332c9-d052-4ec6-83cb-39ce955bc940 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:24.240771 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 620/1259] 10.222.0.87 () {56 vars in 1148 bytes} [Tue Jun 21 18:17:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:17:24.252320 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f139fc-0bdf-49ad-b9d7-dfbd90ed6d54 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:24.313807 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:24.445833 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f139fc-0bdf-49ad-b9d7-dfbd90ed6d54 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:24.506102 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c1c7f-301c-43d9-b99f-105941f202e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:24.507195 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c1c7f-301c-43d9-b99f-105941f202e0 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:24.507709 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c1c7f-301c-43d9-b99f-105941f202e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:24.508141 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c1c7f-301c-43d9-b99f-105941f202e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:24.509647 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2c1c7f-301c-43d9-b99f-105941f202e0 None admin] RBAC: Adding query filter params (role.id=98f0943a610f4a9e9f8de54f6e9f717d, scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb, user.id=ef1d49b36cdc4665b82487be97e9dcd5) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:17:24.510735 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2c1c7f-301c-43d9-b99f-105941f202e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:24.512946 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2c1c7f-301c-43d9-b99f-105941f202e0 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:24.520413 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 640/1260] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:17:24 2022] GET /identity/v3/role_assignments?user.id=ef1d49b36cdc4665b82487be97e9dcd5&scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb&role.id=98f0943a610f4a9e9f8de54f6e9f717d => generated 259 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:17:27.678471 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b978c6-a4f6-445d-911f-b9b49d221fc5 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:27.679313 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b978c6-a4f6-445d-911f-b9b49d221fc5 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:27.679521 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b978c6-a4f6-445d-911f-b9b49d221fc5 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:27.680941 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 621/1261] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:17:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:17:27.689230 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec39762-a8b5-4c85-91de-eefc00a6b6a1 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:27.689901 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec39762-a8b5-4c85-91de-eefc00a6b6a1 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:27.690230 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec39762-a8b5-4c85-91de-eefc00a6b6a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:27.715965 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-8ec39762-a8b5-4c85-91de-eefc00a6b6a1 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:17:27.717277 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8ec39762-a8b5-4c85-91de-eefc00a6b6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:17:27.833543 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 641/1262] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:17:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:17:27.859623 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b978c6-a4f6-445d-911f-b9b49d221fc5 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:27.874255 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:27.997392 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b978c6-a4f6-445d-911f-b9b49d221fc5 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:28.042187 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66211927-9268-4b19-bf05-4bc39da4214a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:28.043475 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66211927-9268-4b19-bf05-4bc39da4214a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:28.043785 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66211927-9268-4b19-bf05-4bc39da4214a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:28.044281 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66211927-9268-4b19-bf05-4bc39da4214a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:28.060571 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66211927-9268-4b19-bf05-4bc39da4214a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:17:28.069249 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-66211927-9268-4b19-bf05-4bc39da4214a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:17:28.069249 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:17:28.069249 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:17:28.069249 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:17:28.069249 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:17:28.069249 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:17:28.069249 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:17:28.069249 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:17:28.069249 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:17:28.069249 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:17:28.069249 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:17:28.069249 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:17:28.069249 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:17:28.069249 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:17:28.069249 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:17:28.069249 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:17:28.071513 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:17:28.071513 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:17:28.071513 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:17:28.071513 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:17:28.071513 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:17:28.071513 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:17:28.071513 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:17:28.071513 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:17:28.071513 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:17:28.071513 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:17:28.071513 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:17:28.071513 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:17:28.071513 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:17:28.071513 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:17:28.071513 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:17:28.071513 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:17:28.073293 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:17:28.073293 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:17:28.073293 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:17:28.073293 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:17:28.073293 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:17:28.073293 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:17:28.073293 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:17:28.073293 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:17:28.073293 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:17:28.073293 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:17:28.073293 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:17:28.073293 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:17:28.073293 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 622/1263] 10.222.0.87 () {56 vars in 1096 bytes} [Tue Jun 21 18:17:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:17:28.082253 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec39762-a8b5-4c85-91de-eefc00a6b6a1 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:28.109179 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:28.224687 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec39762-a8b5-4c85-91de-eefc00a6b6a1 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:28.267679 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79951590-3d3f-4d3a-ae5b-631cc59ef5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:28.268435 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79951590-3d3f-4d3a-ae5b-631cc59ef5e6 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:28.268729 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79951590-3d3f-4d3a-ae5b-631cc59ef5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:28.269006 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79951590-3d3f-4d3a-ae5b-631cc59ef5e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:28.269859 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79951590-3d3f-4d3a-ae5b-631cc59ef5e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:17:28.270182 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79951590-3d3f-4d3a-ae5b-631cc59ef5e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:28.271779 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79951590-3d3f-4d3a-ae5b-631cc59ef5e6 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:28.283804 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 642/1264] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:17:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:17:28.290810 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66211927-9268-4b19-bf05-4bc39da4214a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:28.372668 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:28.419574 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66211927-9268-4b19-bf05-4bc39da4214a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:28.464601 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd00dcf3-1370-430e-98c2-7c087afe2453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:28.465426 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00dcf3-1370-430e-98c2-7c087afe2453 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:28.465753 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00dcf3-1370-430e-98c2-7c087afe2453 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:28.466035 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00dcf3-1370-430e-98c2-7c087afe2453 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:28.479169 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd00dcf3-1370-430e-98c2-7c087afe2453 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:28.480757 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd00dcf3-1370-430e-98c2-7c087afe2453 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:28.501037 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-cd00dcf3-1370-430e-98c2-7c087afe2453 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:17:28.501037 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:17:28.501037 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:17:28.501037 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:17:28.501037 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:17:28.501037 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:17:28.501037 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:17:28.501037 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:17:28.501037 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:17:28.501037 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:17:28.501037 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:17:28.501037 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:17:28.501037 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:17:28.501037 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:17:28.501037 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:17:28.502148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:17:28.502148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:17:28.502148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:17:28.502148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:17:28.502148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:17:28.502148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:17:28.502148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:17:28.502148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:17:28.502148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:17:28.502148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:17:28.502148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:17:28.502148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:17:28.502148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:17:28.502148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:17:28.502148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:17:28.502148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:17:28.502148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:17:28.503357 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:17:28.503357 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:17:28.503357 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:17:28.503357 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:17:28.503357 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 623/1265] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:17:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:17:28.509371 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79951590-3d3f-4d3a-ae5b-631cc59ef5e6 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:28.611257 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:28.641116 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79951590-3d3f-4d3a-ae5b-631cc59ef5e6 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:28.684994 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94380124-c97a-4d02-b59f-91f5863d2972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:28.685806 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94380124-c97a-4d02-b59f-91f5863d2972 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:28.686382 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94380124-c97a-4d02-b59f-91f5863d2972 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:28.687005 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94380124-c97a-4d02-b59f-91f5863d2972 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:28.687550 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94380124-c97a-4d02-b59f-91f5863d2972 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:17:28.687902 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94380124-c97a-4d02-b59f-91f5863d2972 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:28.689384 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94380124-c97a-4d02-b59f-91f5863d2972 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:28.705275 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 643/1266] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:17:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:17:28.712143 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd00dcf3-1370-430e-98c2-7c087afe2453 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:28.843106 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd00dcf3-1370-430e-98c2-7c087afe2453 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:28.871110 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:28.888381 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e5f21-e89c-40d6-abaa-c258c8fa3e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:28.889325 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e5f21-e89c-40d6-abaa-c258c8fa3e8b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:28.889676 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e5f21-e89c-40d6-abaa-c258c8fa3e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:28.889932 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e5f21-e89c-40d6-abaa-c258c8fa3e8b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:28.899526 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021e5f21-e89c-40d6-abaa-c258c8fa3e8b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:28.901149 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021e5f21-e89c-40d6-abaa-c258c8fa3e8b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:28.917529 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-021e5f21-e89c-40d6-abaa-c258c8fa3e8b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 21 18:17:28.917529 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:17:28.917529 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:17:28.917529 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:17:28.917529 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:17:28.917529 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:17:28.917529 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:17:28.917529 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:17:28.917529 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:17:28.917529 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:17:28.917529 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:17:28.917529 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:17:28.917529 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:17:28.917529 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:17:28.917529 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:17:28.917529 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:17:28.919507 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:17:28.919507 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:17:28.919507 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:17:28.919507 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:17:28.919507 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:17:28.919507 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:17:28.919507 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:17:28.919507 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:17:28.919507 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:17:28.919507 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:17:28.919507 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:17:28.919507 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:17:28.919507 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:17:28.919507 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:17:28.919507 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:17:28.919507 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:17:28.919507 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:17:28.919507 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:17:28.921626 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:17:28.921626 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:17:28.921626 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:17:28.921626 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:17:28.921626 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:17:28.921626 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:17:28.921626 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:17:28.921626 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:17:28.921626 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 21 18:17:28.921626 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:17:28.921626 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 624/1267] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:17:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:17:28.926847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94380124-c97a-4d02-b59f-91f5863d2972 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:29.060153 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94380124-c97a-4d02-b59f-91f5863d2972 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:29.104971 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7714ed44-ee61-44a4-b355-03b6c919d24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:29.105703 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7714ed44-ee61-44a4-b355-03b6c919d24a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:29.106038 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7714ed44-ee61-44a4-b355-03b6c919d24a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:29.106283 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7714ed44-ee61-44a4-b355-03b6c919d24a None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:29.107264 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7714ed44-ee61-44a4-b355-03b6c919d24a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:17:29.107584 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7714ed44-ee61-44a4-b355-03b6c919d24a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:29.109109 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7714ed44-ee61-44a4-b355-03b6c919d24a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:29.124365 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:29.137852 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 644/1268] 10.222.0.87 () {56 vars in 1143 bytes} [Tue Jun 21 18:17:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:17:29.144705 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e5f21-e89c-40d6-abaa-c258c8fa3e8b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:29.292684 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e5f21-e89c-40d6-abaa-c258c8fa3e8b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:29.331825 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69747d6-4c9c-4188-82a1-0753704ffce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:29.332657 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69747d6-4c9c-4188-82a1-0753704ffce8 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:29.332945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69747d6-4c9c-4188-82a1-0753704ffce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:29.333216 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69747d6-4c9c-4188-82a1-0753704ffce8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:29.348390 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69747d6-4c9c-4188-82a1-0753704ffce8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:29.349936 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69747d6-4c9c-4188-82a1-0753704ffce8 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:29.370823 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-a69747d6-4c9c-4188-82a1-0753704ffce8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:17:29.370823 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:17:29.370823 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:17:29.370823 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:17:29.370823 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:17:29.370823 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:17:29.370823 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:17:29.370823 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:17:29.370823 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:17:29.370823 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:17:29.370823 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:17:29.370823 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:17:29.370823 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:17:29.370823 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:17:29.370823 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:17:29.376064 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:17:29.376064 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:17:29.376064 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:17:29.376064 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:17:29.376064 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:17:29.376064 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:17:29.376064 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:17:29.376064 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:17:29.376064 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:17:29.376064 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:17:29.376064 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:17:29.376064 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:17:29.376064 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:17:29.376064 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:17:29.376064 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:17:29.376064 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:17:29.376064 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:17:29.377815 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:17:29.377815 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:17:29.377815 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:17:29.377815 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:17:29.377815 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 625/1269] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:17:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:17:29.379330 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7714ed44-ee61-44a4-b355-03b6c919d24a None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:29.510929 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7714ed44-ee61-44a4-b355-03b6c919d24a None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:29.555402 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b836151f-c548-433e-990f-95eb7dfc28dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:29.556122 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836151f-c548-433e-990f-95eb7dfc28dd None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:29.556404 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836151f-c548-433e-990f-95eb7dfc28dd None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:29.556841 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836151f-c548-433e-990f-95eb7dfc28dd None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:29.557730 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b836151f-c548-433e-990f-95eb7dfc28dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:17:29.558061 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b836151f-c548-433e-990f-95eb7dfc28dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:29.559767 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b836151f-c548-433e-990f-95eb7dfc28dd None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:29.575347 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 645/1270] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:17:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:17:29.582469 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69747d6-4c9c-4188-82a1-0753704ffce8 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:29.643476 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:29.780412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69747d6-4c9c-4188-82a1-0753704ffce8 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:29.839876 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c63c10-11d8-4e55-a70d-5ba7a3303c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:29.840740 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c63c10-11d8-4e55-a70d-5ba7a3303c4b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:29.841087 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c63c10-11d8-4e55-a70d-5ba7a3303c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:29.841404 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c63c10-11d8-4e55-a70d-5ba7a3303c4b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:29.850950 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c63c10-11d8-4e55-a70d-5ba7a3303c4b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:29.852252 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c63c10-11d8-4e55-a70d-5ba7a3303c4b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:29.869251 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-05c63c10-11d8-4e55-a70d-5ba7a3303c4b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:17:29.869251 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:17:29.869251 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:17:29.869251 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:17:29.869251 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:17:29.869251 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:17:29.869251 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:17:29.869251 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:17:29.869251 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:17:29.869251 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:17:29.869251 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:17:29.869251 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:17:29.869251 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:17:29.869251 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:17:29.869251 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:17:29.871232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:17:29.871232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:17:29.871232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:17:29.871232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:17:29.871232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:17:29.871232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:17:29.871232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:17:29.871232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:17:29.871232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:17:29.871232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:17:29.871232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:17:29.871232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:17:29.871232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:17:29.871232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:17:29.871232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:17:29.871232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:17:29.871232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:17:29.873432 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:17:29.873432 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:17:29.873432 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:17:29.873432 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:17:29.873432 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:17:29.873432 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:17:29.873432 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:17:29.873432 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:17:29.873432 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 626/1271] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:17:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:17:29.878972 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b836151f-c548-433e-990f-95eb7dfc28dd None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:29.960354 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:30.016775 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b836151f-c548-433e-990f-95eb7dfc28dd None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:30.063251 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44714bbd-0c50-4425-a11c-1b4235ef7fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:30.064001 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44714bbd-0c50-4425-a11c-1b4235ef7fac None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:30.064339 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44714bbd-0c50-4425-a11c-1b4235ef7fac None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:30.064844 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44714bbd-0c50-4425-a11c-1b4235ef7fac None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:30.065612 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44714bbd-0c50-4425-a11c-1b4235ef7fac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:17:30.066046 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44714bbd-0c50-4425-a11c-1b4235ef7fac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:30.067584 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44714bbd-0c50-4425-a11c-1b4235ef7fac None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:30.082617 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 646/1272] 10.222.0.87 () {56 vars in 1148 bytes} [Tue Jun 21 18:17:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:17:30.089557 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c63c10-11d8-4e55-a70d-5ba7a3303c4b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:30.223891 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:30.236741 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c63c10-11d8-4e55-a70d-5ba7a3303c4b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:30.281110 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6bd739-c0a0-48d5-9e89-481ebb6a6c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:30.282360 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6bd739-c0a0-48d5-9e89-481ebb6a6c56 None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:30.282661 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6bd739-c0a0-48d5-9e89-481ebb6a6c56 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:30.282948 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6bd739-c0a0-48d5-9e89-481ebb6a6c56 None admin] PATH_INFO: `/v3/projects/5f30c3ddb5e44e9299ff228fe94c27fb/users/ef1d49b36cdc4665b82487be97e9dcd5/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:30.285481 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6bd739-c0a0-48d5-9e89-481ebb6a6c56 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f30c3ddb5e44e9299ff228fe94c27fb, user_id=ef1d49b36cdc4665b82487be97e9dcd5, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:30.287016 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6bd739-c0a0-48d5-9e89-481ebb6a6c56 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:30.303425 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 627/1273] 10.222.0.87 () {58 vars in 1439 bytes} [Tue Jun 21 18:17:30 2022] PUT /identity/v3/projects/5f30c3ddb5e44e9299ff228fe94c27fb/users/ef1d49b36cdc4665b82487be97e9dcd5/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:17:32.591524 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070874da-371e-4ce1-90b1-0916f15ae96f None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:32.592298 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070874da-371e-4ce1-90b1-0916f15ae96f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:32.592502 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070874da-371e-4ce1-90b1-0916f15ae96f None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:32.595259 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 647/1274] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:17:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:17:32.604147 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9a033-238e-486f-9498-94c51b566ddd None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:32.604964 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9a033-238e-486f-9498-94c51b566ddd None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:32.604964 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9a033-238e-486f-9498-94c51b566ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:32.626887 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b2f9a033-238e-486f-9498-94c51b566ddd None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:17:32.628374 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b2f9a033-238e-486f-9498-94c51b566ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:17:32.762173 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 628/1275] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:17:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:17:32.802845 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070874da-371e-4ce1-90b1-0916f15ae96f None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:32.961148 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070874da-371e-4ce1-90b1-0916f15ae96f None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:32.986833 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:33.015365 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e78593-7c38-4ec0-9971-3f8c1fa0cf04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:33.016467 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e78593-7c38-4ec0-9971-3f8c1fa0cf04 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:33.016755 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e78593-7c38-4ec0-9971-3f8c1fa0cf04 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:33.017040 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e78593-7c38-4ec0-9971-3f8c1fa0cf04 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:33.034023 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e78593-7c38-4ec0-9971-3f8c1fa0cf04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:17:33.044901 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-76e78593-7c38-4ec0-9971-3f8c1fa0cf04 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:17:33.044901 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:17:33.044901 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:17:33.044901 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:17:33.044901 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:17:33.044901 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:17:33.044901 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:17:33.044901 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:17:33.044901 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:17:33.044901 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:17:33.044901 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:17:33.044901 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:17:33.044901 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:17:33.044901 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:17:33.044901 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:17:33.044901 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:17:33.046360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:17:33.046360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:17:33.046360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:17:33.046360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:17:33.046360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:17:33.046360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:17:33.046360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:17:33.046360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:17:33.046360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:17:33.046360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:17:33.046360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:17:33.046360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:17:33.046360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:17:33.046360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:17:33.046360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:17:33.046360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:17:33.047586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:17:33.047586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:17:33.047586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:17:33.047586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:17:33.047586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:17:33.047586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:17:33.047586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:17:33.047586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:17:33.047586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:17:33.047586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:17:33.047586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:17:33.047586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:17:33.047586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 648/1276] 10.222.0.87 () {56 vars in 1096 bytes} [Tue Jun 21 18:17:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:17:33.053202 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f9a033-238e-486f-9498-94c51b566ddd None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:33.188895 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:33.220762 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f9a033-238e-486f-9498-94c51b566ddd None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:33.287844 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40b2f25-aa9d-4fc7-a33d-f636a40afca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:33.288916 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b2f25-aa9d-4fc7-a33d-f636a40afca9 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:33.289333 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b2f25-aa9d-4fc7-a33d-f636a40afca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:33.289767 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b2f25-aa9d-4fc7-a33d-f636a40afca9 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:33.290998 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40b2f25-aa9d-4fc7-a33d-f636a40afca9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:17:33.291528 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40b2f25-aa9d-4fc7-a33d-f636a40afca9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:33.294143 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40b2f25-aa9d-4fc7-a33d-f636a40afca9 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:33.310559 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 629/1277] 10.222.0.87 () {56 vars in 1099 bytes} [Tue Jun 21 18:17:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:17:33.319280 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e78593-7c38-4ec0-9971-3f8c1fa0cf04 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:33.453577 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e78593-7c38-4ec0-9971-3f8c1fa0cf04 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:33.508763 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf83ced-6cec-4eb2-aa0e-82b9bf2d2bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:33.509820 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf83ced-6cec-4eb2-aa0e-82b9bf2d2bab None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:33.510468 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf83ced-6cec-4eb2-aa0e-82b9bf2d2bab None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:33.511085 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf83ced-6cec-4eb2-aa0e-82b9bf2d2bab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:33.526264 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf83ced-6cec-4eb2-aa0e-82b9bf2d2bab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:33.527919 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf83ced-6cec-4eb2-aa0e-82b9bf2d2bab None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:33.549044 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-adf83ced-6cec-4eb2-aa0e-82b9bf2d2bab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:17:33.549044 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:17:33.549044 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:17:33.549044 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:17:33.549044 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:17:33.549044 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:17:33.549044 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:17:33.549044 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:17:33.549044 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:17:33.549044 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:17:33.549044 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:17:33.549044 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:17:33.549044 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:17:33.549044 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:17:33.549044 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:17:33.550861 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:17:33.550861 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:17:33.550861 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:17:33.550861 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:17:33.550861 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:17:33.550861 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:17:33.550861 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:17:33.550861 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:17:33.550861 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:17:33.550861 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:17:33.550861 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:17:33.550861 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:17:33.550861 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:17:33.550861 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:17:33.550861 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:17:33.550861 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:17:33.550861 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:17:33.552596 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:17:33.552596 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:17:33.552596 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:17:33.552596 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:17:33.552596 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 649/1278] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:17:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:17:33.556835 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40b2f25-aa9d-4fc7-a33d-f636a40afca9 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:33.694140 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40b2f25-aa9d-4fc7-a33d-f636a40afca9 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:33.744497 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70fca7d-706d-4e43-a451-d0a2dc10c3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:33.746085 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70fca7d-706d-4e43-a451-d0a2dc10c3c5 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:33.746437 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70fca7d-706d-4e43-a451-d0a2dc10c3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:33.746793 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70fca7d-706d-4e43-a451-d0a2dc10c3c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:33.747625 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70fca7d-706d-4e43-a451-d0a2dc10c3c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:17:33.747992 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70fca7d-706d-4e43-a451-d0a2dc10c3c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:33.749496 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70fca7d-706d-4e43-a451-d0a2dc10c3c5 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:33.765869 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 630/1279] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:17:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:17:33.772376 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf83ced-6cec-4eb2-aa0e-82b9bf2d2bab None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:33.812558 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:33.914541 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf83ced-6cec-4eb2-aa0e-82b9bf2d2bab None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:33.977954 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596b81df-a26d-473f-bba3-9e2f4a1ffc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:33.979338 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b81df-a26d-473f-bba3-9e2f4a1ffc75 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:33.979640 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b81df-a26d-473f-bba3-9e2f4a1ffc75 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:33.979935 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b81df-a26d-473f-bba3-9e2f4a1ffc75 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:33.993620 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596b81df-a26d-473f-bba3-9e2f4a1ffc75 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:33.995843 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596b81df-a26d-473f-bba3-9e2f4a1ffc75 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:34.014388 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-596b81df-a26d-473f-bba3-9e2f4a1ffc75 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 21 18:17:34.014388 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:17:34.014388 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:17:34.014388 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:17:34.014388 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:17:34.014388 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:17:34.014388 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:17:34.014388 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:17:34.014388 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:17:34.014388 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:17:34.014388 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:17:34.014388 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:17:34.014388 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:17:34.014388 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:17:34.014388 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:17:34.014388 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:17:34.016253 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:17:34.016253 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:17:34.016253 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:17:34.016253 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:17:34.016253 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:17:34.016253 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:17:34.016253 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:17:34.016253 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:17:34.016253 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:17:34.016253 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:17:34.016253 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:17:34.016253 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:17:34.016253 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:17:34.016253 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:17:34.016253 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:17:34.016253 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:17:34.016253 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:17:34.016253 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:17:34.017657 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:17:34.017657 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:17:34.017657 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:17:34.017657 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:17:34.017657 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:17:34.017657 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:17:34.017657 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:17:34.017657 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:17:34.017657 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 21 18:17:34.017657 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:17:34.017657 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 650/1280] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:17:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:17:34.024150 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70fca7d-706d-4e43-a451-d0a2dc10c3c5 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:34.029403 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:34.177114 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70fca7d-706d-4e43-a451-d0a2dc10c3c5 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:34.228425 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4e884a-a024-458b-ac2b-7461b0490e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:34.229331 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e884a-a024-458b-ac2b-7461b0490e94 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:34.229698 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e884a-a024-458b-ac2b-7461b0490e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:34.230031 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e884a-a024-458b-ac2b-7461b0490e94 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:34.231044 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4e884a-a024-458b-ac2b-7461b0490e94 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:17:34.231288 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4e884a-a024-458b-ac2b-7461b0490e94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:34.232778 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4e884a-a024-458b-ac2b-7461b0490e94 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:34.255405 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 631/1281] 10.222.0.87 () {56 vars in 1143 bytes} [Tue Jun 21 18:17:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:17:34.262381 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596b81df-a26d-473f-bba3-9e2f4a1ffc75 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:34.390871 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:34.404383 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596b81df-a26d-473f-bba3-9e2f4a1ffc75 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:34.453091 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f02e8-0372-4ded-8822-e6d2d975f87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:34.453883 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f02e8-0372-4ded-8822-e6d2d975f87e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:34.454223 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f02e8-0372-4ded-8822-e6d2d975f87e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:34.454551 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f02e8-0372-4ded-8822-e6d2d975f87e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:34.560378 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f02e8-0372-4ded-8822-e6d2d975f87e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:34.562439 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f02e8-0372-4ded-8822-e6d2d975f87e None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:34.720229 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:34.742883 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 651/1282] 10.222.0.87 () {58 vars in 1282 bytes} [Tue Jun 21 18:17:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:17:34.749963 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4e884a-a024-458b-ac2b-7461b0490e94 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:34.826429 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:34.878443 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4e884a-a024-458b-ac2b-7461b0490e94 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:34.925866 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f6034e-e2d0-495f-925e-d4243ab6fc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:34.926780 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6034e-e2d0-495f-925e-d4243ab6fc21 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:34.927065 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6034e-e2d0-495f-925e-d4243ab6fc21 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:34.927410 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6034e-e2d0-495f-925e-d4243ab6fc21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:34.941878 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f6034e-e2d0-495f-925e-d4243ab6fc21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:34.943426 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f6034e-e2d0-495f-925e-d4243ab6fc21 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:34.965289 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-e5f6034e-e2d0-495f-925e-d4243ab6fc21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:17:34.965289 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:17:34.965289 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:17:34.965289 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:17:34.965289 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:17:34.965289 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:17:34.965289 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:17:34.965289 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:17:34.965289 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:17:34.965289 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:17:34.965289 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:17:34.965289 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:17:34.965289 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:17:34.965289 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:17:34.965289 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:17:34.966638 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:17:34.966638 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:17:34.966638 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:17:34.966638 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:17:34.966638 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:17:34.966638 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:17:34.966638 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:17:34.966638 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:17:34.966638 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:17:34.966638 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:17:34.966638 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:17:34.966638 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:17:34.966638 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:17:34.966638 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:17:34.966638 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:17:34.966638 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:17:34.966638 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:17:34.967865 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:17:34.967865 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:17:34.967865 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:17:34.967865 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:17:34.967865 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 632/1283] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:17:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:17:34.976332 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f02e8-0372-4ded-8822-e6d2d975f87e None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:35.160384 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f02e8-0372-4ded-8822-e6d2d975f87e None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:35.204531 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47427d4-159a-44a9-b72d-6933f9e95135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:35.205212 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47427d4-159a-44a9-b72d-6933f9e95135 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:35.205503 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47427d4-159a-44a9-b72d-6933f9e95135 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:35.205819 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47427d4-159a-44a9-b72d-6933f9e95135 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:35.206556 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47427d4-159a-44a9-b72d-6933f9e95135 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:17:35.206869 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47427d4-159a-44a9-b72d-6933f9e95135 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:35.208402 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47427d4-159a-44a9-b72d-6933f9e95135 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:35.227891 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 652/1284] 10.222.0.87 () {56 vars in 1109 bytes} [Tue Jun 21 18:17:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:17:35.234196 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f6034e-e2d0-495f-925e-d4243ab6fc21 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:35.376987 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f6034e-e2d0-495f-925e-d4243ab6fc21 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:35.394689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:35.420118 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec0788-b32a-4dfe-bd33-01ce816ca00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:35.420946 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec0788-b32a-4dfe-bd33-01ce816ca00b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:35.421233 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec0788-b32a-4dfe-bd33-01ce816ca00b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:35.421511 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec0788-b32a-4dfe-bd33-01ce816ca00b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:35.429563 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eec0788-b32a-4dfe-bd33-01ce816ca00b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:35.431187 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eec0788-b32a-4dfe-bd33-01ce816ca00b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:35.447420 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-3eec0788-b32a-4dfe-bd33-01ce816ca00b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:17:35.447420 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:17:35.447420 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:17:35.447420 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:17:35.447420 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:17:35.447420 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:17:35.447420 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:17:35.447420 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:17:35.447420 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:17:35.447420 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:17:35.447420 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:17:35.447420 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:17:35.447420 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:17:35.447420 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:17:35.447420 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:17:35.449412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:17:35.449412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:17:35.449412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:17:35.449412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:17:35.449412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:17:35.449412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:17:35.449412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:17:35.449412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:17:35.449412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 21 18:17:35.449412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:17:35.449412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:17:35.449412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:17:35.449412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:17:35.449412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:17:35.449412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:17:35.449412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:17:35.449412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:17:35.451231 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:17:35.451231 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:17:35.451231 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:17:35.451231 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:17:35.451231 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:17:35.451231 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:17:35.451231 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:17:35.451231 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 21 18:17:35.451231 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 633/1285] 10.222.0.87 () {56 vars in 1111 bytes} [Tue Jun 21 18:17:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:17:35.456333 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47427d4-159a-44a9-b72d-6933f9e95135 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:35.522846 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:35.593573 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47427d4-159a-44a9-b72d-6933f9e95135 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:35.640783 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b1e751-00e7-4192-8892-9f55384a9e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:35.641518 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1e751-00e7-4192-8892-9f55384a9e9e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:35.641946 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1e751-00e7-4192-8892-9f55384a9e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:35.642223 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1e751-00e7-4192-8892-9f55384a9e9e None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:35.643626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b1e751-00e7-4192-8892-9f55384a9e9e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:17:35.644110 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b1e751-00e7-4192-8892-9f55384a9e9e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:35.645624 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b1e751-00e7-4192-8892-9f55384a9e9e None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:35.660261 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 653/1286] 10.222.0.87 () {56 vars in 1148 bytes} [Tue Jun 21 18:17:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:17:35.667098 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec0788-b32a-4dfe-bd33-01ce816ca00b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:35.797506 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec0788-b32a-4dfe-bd33-01ce816ca00b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:35.845194 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e6e9b0-177e-4d23-b18b-76e796491fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:35.846002 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e6e9b0-177e-4d23-b18b-76e796491fbb None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:35.846364 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e6e9b0-177e-4d23-b18b-76e796491fbb None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:35.846654 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e6e9b0-177e-4d23-b18b-76e796491fbb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:35.847529 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e6e9b0-177e-4d23-b18b-76e796491fbb None admin] RBAC: Adding query filter params (role.id=98f0943a610f4a9e9f8de54f6e9f717d, scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb, user.id=ef1d49b36cdc4665b82487be97e9dcd5) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:17:35.847850 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e6e9b0-177e-4d23-b18b-76e796491fbb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:35.849377 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e6e9b0-177e-4d23-b18b-76e796491fbb None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:35.857778 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 634/1287] 10.222.0.87 () {56 vars in 1374 bytes} [Tue Jun 21 18:17:35 2022] GET /identity/v3/role_assignments?user.id=ef1d49b36cdc4665b82487be97e9dcd5&scope.project.id=5f30c3ddb5e44e9299ff228fe94c27fb&role.id=98f0943a610f4a9e9f8de54f6e9f717d => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:17:38.306470 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3a677-a1ee-45c6-9974-7391811cb97a None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:38.307168 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3a677-a1ee-45c6-9974-7391811cb97a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:38.307315 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3a677-a1ee-45c6-9974-7391811cb97a None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:38.309541 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 654/1288] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:17:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:17:38.317563 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7063a468-4285-4038-9923-5ee2d6cbd4c6 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:38.318394 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7063a468-4285-4038-9923-5ee2d6cbd4c6 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:38.318394 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7063a468-4285-4038-9923-5ee2d6cbd4c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:38.340054 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-7063a468-4285-4038-9923-5ee2d6cbd4c6 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:17:38.341965 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7063a468-4285-4038-9923-5ee2d6cbd4c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:17:38.414627 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:38.477962 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 635/1289] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:17:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:17:38.518554 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e3a677-a1ee-45c6-9974-7391811cb97a None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:38.691516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e3a677-a1ee-45c6-9974-7391811cb97a None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:38.707040 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:38.740771 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34185d81-ed27-41e0-bd9a-59e31223484f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:38.742430 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34185d81-ed27-41e0-bd9a-59e31223484f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:38.742718 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34185d81-ed27-41e0-bd9a-59e31223484f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:38.742997 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34185d81-ed27-41e0-bd9a-59e31223484f None admin] PATH_INFO: `/v3/services/placement` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:38.743699 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34185d81-ed27-41e0-bd9a-59e31223484f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:38.745241 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34185d81-ed27-41e0-bd9a-59e31223484f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:38.763298 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-34185d81-ed27-41e0-bd9a-59e31223484f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 21 18:17:38.763298 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:17:38.763298 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:17:38.763298 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:17:38.763298 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:17:38.763298 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:17:38.763298 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:17:38.763298 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:17:38.763298 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 21 18:17:38.763298 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:17:38.763298 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:17:38.763298 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:17:38.763298 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:17:38.763298 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:17:38.763298 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:17:38.763298 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:17:38.763298 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:17:38.765578 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:17:38.765578 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:17:38.765578 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:17:38.765578 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:17:38.765578 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:17:38.765578 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:17:38.765578 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:17:38.765578 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:17:38.765578 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:17:38.765578 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:17:38.765578 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:17:38.765578 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:17:38.765578 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:17:38.765578 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:17:38.765578 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:17:38.765578 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:17:38.767584 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:17:38.767584 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:17:38.767584 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:17:38.767584 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:17:38.767584 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:17:38.767584 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:17:38.767584 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:17:38.767584 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:17:38.767584 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:17:38.767584 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:17:38.767584 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:17:38.767584 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:17:38.767584 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 21 18:17:38.767584 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:17:38.767584 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 655/1290] 10.222.0.87 () {56 vars in 1117 bytes} [Tue Jun 21 18:17:38 2022] GET /identity/v3/services/placement => generated 90 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:17:38.772427 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7063a468-4285-4038-9923-5ee2d6cbd4c6 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:38.921190 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7063a468-4285-4038-9923-5ee2d6cbd4c6 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:38.981333 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274cff89-8d92-4a92-838a-feb974532037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:38.982323 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274cff89-8d92-4a92-838a-feb974532037 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:38.982764 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274cff89-8d92-4a92-838a-feb974532037 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:38.983192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274cff89-8d92-4a92-838a-feb974532037 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:38.984344 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274cff89-8d92-4a92-838a-feb974532037 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:17:38.984822 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274cff89-8d92-4a92-838a-feb974532037 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:38.987189 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274cff89-8d92-4a92-838a-feb974532037 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:38.996656 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 636/1291] 10.222.0.87 () {56 vars in 1116 bytes} [Tue Jun 21 18:17:38 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:17:39.007435 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34185d81-ed27-41e0-bd9a-59e31223484f None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:39.225793 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34185d81-ed27-41e0-bd9a-59e31223484f None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:39.277451 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e8ecee-67db-4d3d-829f-8e6417d629f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:39.278390 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8ecee-67db-4d3d-829f-8e6417d629f1 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:39.278390 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8ecee-67db-4d3d-829f-8e6417d629f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:39.278629 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8ecee-67db-4d3d-829f-8e6417d629f1 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:39.279412 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e8ecee-67db-4d3d-829f-8e6417d629f1 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:17:39.279731 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e8ecee-67db-4d3d-829f-8e6417d629f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:39.281153 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e8ecee-67db-4d3d-829f-8e6417d629f1 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:39.287236 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 656/1292] 10.222.0.87 () {56 vars in 1116 bytes} [Tue Jun 21 18:17:39 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:17:41.750212 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3578161-f29f-4e63-9a59-677432f6909f None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:41.750822 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3578161-f29f-4e63-9a59-677432f6909f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:41.750822 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3578161-f29f-4e63-9a59-677432f6909f None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:41.752293 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 637/1293] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:17:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:17:41.758448 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edec17d-5026-449f-85a7-a477510e0a2e None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:41.759042 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edec17d-5026-449f-85a7-a477510e0a2e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:41.759160 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edec17d-5026-449f-85a7-a477510e0a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:41.778746 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-6edec17d-5026-449f-85a7-a477510e0a2e None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:17:41.780051 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-6edec17d-5026-449f-85a7-a477510e0a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:17:41.819513 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:41.867755 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 657/1294] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:17:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:17:41.892419 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3578161-f29f-4e63-9a59-677432f6909f None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:41.931195 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:42.020387 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3578161-f29f-4e63-9a59-677432f6909f None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:42.066072 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4305a828-96e3-4557-9ae7-1b31c2ffb51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:42.066797 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4305a828-96e3-4557-9ae7-1b31c2ffb51c None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:42.067085 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4305a828-96e3-4557-9ae7-1b31c2ffb51c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:42.067366 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4305a828-96e3-4557-9ae7-1b31c2ffb51c None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:42.068118 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4305a828-96e3-4557-9ae7-1b31c2ffb51c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:42.069597 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4305a828-96e3-4557-9ae7-1b31c2ffb51c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:42.085286 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 638/1295] 10.222.0.87 () {60 vars in 1141 bytes} [Tue Jun 21 18:17:41 2022] POST /identity/v3/services => generated 246 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:17:44.362743 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103fe45-4d7d-4fc0-b6ec-29febfa4ddb0 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:44.363704 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103fe45-4d7d-4fc0-b6ec-29febfa4ddb0 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:44.363892 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103fe45-4d7d-4fc0-b6ec-29febfa4ddb0 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:44.366911 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 658/1296] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:17:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:17:44.375664 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd699875-5e9a-4eae-9ab0-fde1f936c2b6 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:44.376524 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd699875-5e9a-4eae-9ab0-fde1f936c2b6 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:44.376524 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd699875-5e9a-4eae-9ab0-fde1f936c2b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:44.410266 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-fd699875-5e9a-4eae-9ab0-fde1f936c2b6 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:17:44.412397 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fd699875-5e9a-4eae-9ab0-fde1f936c2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:17:44.555991 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 639/1297] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:17:44 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:17:44.582842 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1103fe45-4d7d-4fc0-b6ec-29febfa4ddb0 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:44.723895 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1103fe45-4d7d-4fc0-b6ec-29febfa4ddb0 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:44.766964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db63c4-1379-4a7b-8070-bb4b42b41947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:44.767886 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db63c4-1379-4a7b-8070-bb4b42b41947 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:44.768169 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db63c4-1379-4a7b-8070-bb4b42b41947 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:44.768446 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db63c4-1379-4a7b-8070-bb4b42b41947 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:44.769115 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6db63c4-1379-4a7b-8070-bb4b42b41947 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:44.770765 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6db63c4-1379-4a7b-8070-bb4b42b41947 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:44.784544 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-a6db63c4-1379-4a7b-8070-bb4b42b41947 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 21 18:17:44.784544 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:17:44.784544 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:17:44.784544 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:17:44.784544 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:17:44.784544 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:17:44.784544 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:17:44.784544 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:17:44.784544 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 21 18:17:44.784544 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:17:44.784544 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:17:44.784544 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:17:44.784544 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:17:44.784544 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:17:44.784544 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:17:44.784544 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:17:44.784544 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:17:44.786902 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:17:44.786902 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:17:44.786902 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:17:44.786902 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:17:44.786902 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:17:44.786902 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:17:44.786902 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:17:44.786902 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:17:44.786902 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:17:44.786902 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:17:44.786902 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:17:44.786902 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:17:44.786902 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:17:44.786902 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:17:44.786902 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:17:44.786902 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:17:44.788896 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:17:44.788896 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:17:44.788896 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:17:44.788896 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:17:44.788896 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:17:44.788896 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:17:44.788896 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:17:44.788896 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:17:44.788896 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:17:44.788896 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:17:44.788896 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:17:44.788896 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:17:44.788896 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 21 18:17:44.788896 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:17:44.788896 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 659/1298] 10.222.0.87 () {56 vars in 1117 bytes} [Tue Jun 21 18:17:44 2022] GET /identity/v3/services/placement => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:17:44.793149 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd699875-5e9a-4eae-9ab0-fde1f936c2b6 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:44.827499 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:44.989726 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd699875-5e9a-4eae-9ab0-fde1f936c2b6 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:45.051766 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec933fc-5491-404a-b7f2-0085e66a8afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:45.052910 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec933fc-5491-404a-b7f2-0085e66a8afb None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:45.053376 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec933fc-5491-404a-b7f2-0085e66a8afb None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:45.053901 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec933fc-5491-404a-b7f2-0085e66a8afb None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:45.059190 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec933fc-5491-404a-b7f2-0085e66a8afb None admin] RBAC: Adding query filter params (name=placement) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:17:45.059190 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec933fc-5491-404a-b7f2-0085e66a8afb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:45.059190 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec933fc-5491-404a-b7f2-0085e66a8afb None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:45.069572 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 640/1299] 10.222.0.87 () {56 vars in 1116 bytes} [Tue Jun 21 18:17:44 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:17:45.079607 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db63c4-1379-4a7b-8070-bb4b42b41947 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:45.132530 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:45.244148 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db63c4-1379-4a7b-8070-bb4b42b41947 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:45.313101 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6fc746-1bf0-4d41-a933-62a7c55569b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:45.314336 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6fc746-1bf0-4d41-a933-62a7c55569b3 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:45.314826 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6fc746-1bf0-4d41-a933-62a7c55569b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:45.315327 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6fc746-1bf0-4d41-a933-62a7c55569b3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:45.316945 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6fc746-1bf0-4d41-a933-62a7c55569b3 None admin] RBAC: Adding query filter params (interface=public, service_id=62a404f5b65241578f4f02361ab0b08e, region_id=RegionOne) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:17:45.317501 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6fc746-1bf0-4d41-a933-62a7c55569b3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:45.320337 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6fc746-1bf0-4d41-a933-62a7c55569b3 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:45.331754 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 660/1300] 10.222.0.87 () {56 vars in 1251 bytes} [Tue Jun 21 18:17:45 2022] GET /identity/v3/endpoints?service_id=62a404f5b65241578f4f02361ab0b08e&interface=public®ion_id=RegionOne => generated 194 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:17:45.339420 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec933fc-5491-404a-b7f2-0085e66a8afb None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:45.492491 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:45.518339 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec933fc-5491-404a-b7f2-0085e66a8afb None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:45.556921 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4974fc4b-aa30-42ce-beb1-f9b14ab7df54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:45.557902 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974fc4b-aa30-42ce-beb1-f9b14ab7df54 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:45.558243 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974fc4b-aa30-42ce-beb1-f9b14ab7df54 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:45.558581 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974fc4b-aa30-42ce-beb1-f9b14ab7df54 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:45.559346 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4974fc4b-aa30-42ce-beb1-f9b14ab7df54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:45.560824 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4974fc4b-aa30-42ce-beb1-f9b14ab7df54 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:45.569063 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 641/1301] 10.222.0.87 () {56 vars in 1087 bytes} [Tue Jun 21 18:17:45 2022] GET /identity/v3/services => generated 2191 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:17:48.206562 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3988df4e-d6a3-4949-bd71-baa0e68446fa None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:48.207399 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3988df4e-d6a3-4949-bd71-baa0e68446fa None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:48.207399 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3988df4e-d6a3-4949-bd71-baa0e68446fa None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:48.208705 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 661/1302] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:17:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:17:48.214813 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705d325-d2e5-4d7b-8a02-0bbb5cd92e16 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:48.215435 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705d325-d2e5-4d7b-8a02-0bbb5cd92e16 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:48.215435 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705d325-d2e5-4d7b-8a02-0bbb5cd92e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:48.234615 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-8705d325-d2e5-4d7b-8a02-0bbb5cd92e16 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:17:48.235954 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8705d325-d2e5-4d7b-8a02-0bbb5cd92e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:17:48.325214 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 642/1303] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:17:48 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:17:48.350730 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3988df4e-d6a3-4949-bd71-baa0e68446fa None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:48.478666 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:48.491401 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3988df4e-d6a3-4949-bd71-baa0e68446fa None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:48.560444 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21248259-264f-4cd0-9e90-9689b77c3a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:48.561923 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21248259-264f-4cd0-9e90-9689b77c3a68 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:48.562418 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21248259-264f-4cd0-9e90-9689b77c3a68 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:48.562913 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21248259-264f-4cd0-9e90-9689b77c3a68 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:48.564181 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21248259-264f-4cd0-9e90-9689b77c3a68 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:48.566780 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21248259-264f-4cd0-9e90-9689b77c3a68 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:48.590494 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-21248259-264f-4cd0-9e90-9689b77c3a68 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 21 18:17:48.590494 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:17:48.590494 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:17:48.590494 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:17:48.590494 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:17:48.590494 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:17:48.590494 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:17:48.590494 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:17:48.590494 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 21 18:17:48.590494 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:17:48.590494 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:17:48.590494 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:17:48.590494 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:17:48.590494 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:17:48.590494 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:17:48.590494 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:17:48.590494 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:17:48.591828 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:17:48.591828 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:17:48.591828 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:17:48.591828 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:17:48.591828 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:17:48.591828 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:17:48.591828 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:17:48.591828 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:17:48.591828 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:17:48.591828 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:17:48.591828 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:17:48.591828 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:17:48.591828 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:17:48.591828 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:17:48.591828 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:17:48.591828 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:17:48.593086 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:17:48.593086 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 21 18:17:48.593086 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:17:48.593086 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:17:48.593086 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:17:48.593086 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:17:48.593086 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:17:48.593086 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:17:48.593086 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:17:48.593086 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:17:48.593086 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:17:48.593086 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:17:48.593086 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 21 18:17:48.593086 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 21 18:17:48.593086 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 662/1304] 10.222.0.87 () {56 vars in 1117 bytes} [Tue Jun 21 18:17:48 2022] GET /identity/v3/services/placement => generated 90 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:17:48.599661 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8705d325-d2e5-4d7b-8a02-0bbb5cd92e16 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:48.703522 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:48.732134 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8705d325-d2e5-4d7b-8a02-0bbb5cd92e16 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:48.778874 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896b8f3e-1bb6-4c51-9d41-17af6143e04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:48.779574 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b8f3e-1bb6-4c51-9d41-17af6143e04a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:48.779863 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b8f3e-1bb6-4c51-9d41-17af6143e04a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:48.780145 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b8f3e-1bb6-4c51-9d41-17af6143e04a None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:48.780895 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896b8f3e-1bb6-4c51-9d41-17af6143e04a None admin] RBAC: Adding query filter params (name=placement) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:17:48.781211 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896b8f3e-1bb6-4c51-9d41-17af6143e04a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:48.782697 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896b8f3e-1bb6-4c51-9d41-17af6143e04a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:48.789684 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 643/1305] 10.222.0.87 () {56 vars in 1116 bytes} [Tue Jun 21 18:17:48 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:17:48.799471 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21248259-264f-4cd0-9e90-9689b77c3a68 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:17:48.946065 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21248259-264f-4cd0-9e90-9689b77c3a68 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:17:49.014205 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9bff41-cfaa-4b64-a719-d64dcfe5eb1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:17:49.015225 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9bff41-cfaa-4b64-a719-d64dcfe5eb1d None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:17:49.015729 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9bff41-cfaa-4b64-a719-d64dcfe5eb1d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:17:49.016249 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9bff41-cfaa-4b64-a719-d64dcfe5eb1d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:17:49.017705 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9bff41-cfaa-4b64-a719-d64dcfe5eb1d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:17:49.020287 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9bff41-cfaa-4b64-a719-d64dcfe5eb1d None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:17:49.054321 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 663/1306] 10.222.0.87 () {60 vars in 1144 bytes} [Tue Jun 21 18:17:48 2022] POST /identity/v3/endpoints => generated 331 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:17:58.785564 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=30 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:17:59.045492 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:19:15.918040 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4393e-d97b-4c3c-a6ef-2e4c9357a72c None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:15.919871 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4393e-d97b-4c3c-a6ef-2e4c9357a72c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:15.920710 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4393e-d97b-4c3c-a6ef-2e4c9357a72c None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:15.930959 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 644/1307] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:19:15 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:19:15.934062 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172342d5-9d54-45da-a5b3-874cc35003a5 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:15.934954 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172342d5-9d54-45da-a5b3-874cc35003a5 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:15.935667 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172342d5-9d54-45da-a5b3-874cc35003a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:16.127880 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-172342d5-9d54-45da-a5b3-874cc35003a5 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:19:16.130687 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-172342d5-9d54-45da-a5b3-874cc35003a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:16.349673 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 664/1308] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:19:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 418 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:19:16.474407 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a298fa1-c987-46fc-84ef-7b1684a5c72c None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:16.474407 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a298fa1-c987-46fc-84ef-7b1684a5c72c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:16.474407 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a298fa1-c987-46fc-84ef-7b1684a5c72c None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:16.481542 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 645/1309] 10.222.0.87 () {54 vars in 964 bytes} [Tue Jun 21 18:19:16 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:19:16.496846 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a46d51-2f3b-41d9-9629-147956ccb3bb None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:16.496846 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a46d51-2f3b-41d9-9629-147956ccb3bb None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:16.496846 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a46d51-2f3b-41d9-9629-147956ccb3bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:16.703319 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-a2a46d51-2f3b-41d9-9629-147956ccb3bb None None] MFA Rules not processed for user `480e2529015c4c2fb4d6cd1985009fde`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:19:16.705755 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a2a46d51-2f3b-41d9-9629-147956ccb3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:16.755204 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:19:16.893630 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 665/1310] 10.222.0.87 () {58 vars in 1061 bytes} [Tue Jun 21 18:19:16 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 403 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:19:16.908789 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a298fa1-c987-46fc-84ef-7b1684a5c72c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:19:16.912278 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9a298fa1-c987-46fc-84ef-7b1684a5c72c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:17.141293 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a298fa1-c987-46fc-84ef-7b1684a5c72c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:19:17.299142 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4df167-13c6-4828-8740-0e241afd5d02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:19:17.300368 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4df167-13c6-4828-8740-0e241afd5d02 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:17.300709 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4df167-13c6-4828-8740-0e241afd5d02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:17.301186 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4df167-13c6-4828-8740-0e241afd5d02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:17.304131 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0b4df167-13c6-4828-8740-0e241afd5d02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:17.387541 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:19:17.501471 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4df167-13c6-4828-8740-0e241afd5d02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:19:17.503009 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4df167-13c6-4828-8740-0e241afd5d02 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:19:17.528863 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 646/1311] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:19:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 621 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:19:20.930229 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea30c42-19c9-44d5-83a4-18dbc0eb8421 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:20.931064 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea30c42-19c9-44d5-83a4-18dbc0eb8421 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:20.931064 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea30c42-19c9-44d5-83a4-18dbc0eb8421 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:20.933740 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 666/1312] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:19:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:19:20.939747 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760e1ba-5754-47fa-8fb6-031566c7c635 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:20.940374 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760e1ba-5754-47fa-8fb6-031566c7c635 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:20.940374 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760e1ba-5754-47fa-8fb6-031566c7c635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:21.034161 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-6760e1ba-5754-47fa-8fb6-031566c7c635 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:19:21.035415 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6760e1ba-5754-47fa-8fb6-031566c7c635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:21.157864 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 647/1313] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:19:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:19:21.198448 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea30c42-19c9-44d5-83a4-18dbc0eb8421 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:19:21.200378 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-eea30c42-19c9-44d5-83a4-18dbc0eb8421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:21.364691 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea30c42-19c9-44d5-83a4-18dbc0eb8421 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:19:21.478368 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:19:21.482218 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d7d1c-d1a0-4d2a-b453-bc05f8f8ca4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:19:21.483232 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d7d1c-d1a0-4d2a-b453-bc05f8f8ca4f service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:21.483758 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d7d1c-d1a0-4d2a-b453-bc05f8f8ca4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:21.484319 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d7d1c-d1a0-4d2a-b453-bc05f8f8ca4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:21.486881 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-bf7d7d1c-d1a0-4d2a-b453-bc05f8f8ca4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:21.643656 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7d7d1c-d1a0-4d2a-b453-bc05f8f8ca4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:19:21.645137 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7d7d1c-d1a0-4d2a-b453-bc05f8f8ca4f service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:19:21.668240 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 667/1314] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:19:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:19:24.378454 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54b241-955d-42a8-9d01-a92be2078e4c None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:24.379067 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54b241-955d-42a8-9d01-a92be2078e4c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:24.379401 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54b241-955d-42a8-9d01-a92be2078e4c None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:24.381921 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 648/1315] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:19:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:19:24.394850 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f6a33-310c-4018-b622-bb21e3f92679 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:24.395496 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f6a33-310c-4018-b622-bb21e3f92679 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:24.395496 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f6a33-310c-4018-b622-bb21e3f92679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:24.479742 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-024f6a33-310c-4018-b622-bb21e3f92679 None None] MFA Rules not processed for user `786bcdb4dc274deb8280030ff16c6d0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:19:24.481265 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-024f6a33-310c-4018-b622-bb21e3f92679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:24.560646 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:19:24.625735 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 668/1316] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:19:24 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:19:24.667209 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c54b241-955d-42a8-9d01-a92be2078e4c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:19:24.669074 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9c54b241-955d-42a8-9d01-a92be2078e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:24.681505 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:19:24.854828 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c54b241-955d-42a8-9d01-a92be2078e4c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:19:24.967632 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535459de-ed09-4da2-82fc-d42cf5822a43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:19:24.968562 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535459de-ed09-4da2-82fc-d42cf5822a43 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:24.968990 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535459de-ed09-4da2-82fc-d42cf5822a43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:24.969374 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535459de-ed09-4da2-82fc-d42cf5822a43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:24.971404 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-535459de-ed09-4da2-82fc-d42cf5822a43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:25.082481 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:19:25.169070 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535459de-ed09-4da2-82fc-d42cf5822a43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:19:25.171236 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535459de-ed09-4da2-82fc-d42cf5822a43 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:19:25.202378 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 649/1317] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:19:24 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:19:28.778873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f73fb-f27d-4d1f-b91a-d19b4b9dc906 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:28.779679 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f73fb-f27d-4d1f-b91a-d19b4b9dc906 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:28.779679 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f73fb-f27d-4d1f-b91a-d19b4b9dc906 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:28.780893 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 669/1318] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:19:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:19:28.788375 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d5300-3eff-49ea-9f7a-9a5501be80ec None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:28.788980 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d5300-3eff-49ea-9f7a-9a5501be80ec None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:28.789343 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d5300-3eff-49ea-9f7a-9a5501be80ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:28.899802 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-772d5300-3eff-49ea-9f7a-9a5501be80ec None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:19:28.901100 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-772d5300-3eff-49ea-9f7a-9a5501be80ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:29.098308 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 650/1319] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:19:28 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:19:29.146522 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73f73fb-f27d-4d1f-b91a-d19b4b9dc906 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:19:29.151484 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d73f73fb-f27d-4d1f-b91a-d19b4b9dc906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:29.330834 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73f73fb-f27d-4d1f-b91a-d19b4b9dc906 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:19:29.443939 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fdad34-5186-4cf9-bba5-48a9beed4dcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:19:29.444627 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fdad34-5186-4cf9-bba5-48a9beed4dcd service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:29.444912 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fdad34-5186-4cf9-bba5-48a9beed4dcd service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:29.445231 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fdad34-5186-4cf9-bba5-48a9beed4dcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:29.447424 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e1fdad34-5186-4cf9-bba5-48a9beed4dcd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:29.457067 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:19:29.616938 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fdad34-5186-4cf9-bba5-48a9beed4dcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:19:29.619075 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fdad34-5186-4cf9-bba5-48a9beed4dcd service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:19:29.658978 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 670/1320] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:19:29 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:19:33.437492 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94f7a2-4752-4b7d-a96c-d68b21a92a24 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:33.438843 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94f7a2-4752-4b7d-a96c-d68b21a92a24 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:33.438843 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94f7a2-4752-4b7d-a96c-d68b21a92a24 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:33.441372 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 651/1321] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:19:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:19:33.450299 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2064f3e7-3205-435b-8918-e18148097d85 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:33.451507 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2064f3e7-3205-435b-8918-e18148097d85 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:33.451959 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2064f3e7-3205-435b-8918-e18148097d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:33.594068 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-2064f3e7-3205-435b-8918-e18148097d85 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:19:33.595844 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-2064f3e7-3205-435b-8918-e18148097d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:33.679718 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:19:33.768568 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 671/1322] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:19:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:19:33.852710 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e95f0d-6151-4c60-82a1-8ce899c80ef7 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:33.853130 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e95f0d-6151-4c60-82a1-8ce899c80ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:33.853464 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e95f0d-6151-4c60-82a1-8ce899c80ef7 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:33.855535 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 652/1323] 10.222.0.87 () {54 vars in 964 bytes} [Tue Jun 21 18:19:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:19:33.868126 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ea18f-2239-499e-a691-ef4437d1a932 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:33.868616 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ea18f-2239-499e-a691-ef4437d1a932 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:33.869116 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ea18f-2239-499e-a691-ef4437d1a932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:34.020270 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-7f2ea18f-2239-499e-a691-ef4437d1a932 None None] MFA Rules not processed for user `480e2529015c4c2fb4d6cd1985009fde`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:19:34.022141 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7f2ea18f-2239-499e-a691-ef4437d1a932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:34.155168 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 672/1324] 10.222.0.87 () {58 vars in 1061 bytes} [Tue Jun 21 18:19:33 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:19:34.166607 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e95f0d-6151-4c60-82a1-8ce899c80ef7 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:19:34.169851 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-55e95f0d-6151-4c60-82a1-8ce899c80ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:34.213986 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:19:34.407777 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e95f0d-6151-4c60-82a1-8ce899c80ef7 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:19:34.528877 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8fb3d3-1dce-40c1-85b9-fea50213a32c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:19:34.530414 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8fb3d3-1dce-40c1-85b9-fea50213a32c service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:34.530878 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8fb3d3-1dce-40c1-85b9-fea50213a32c service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:34.531361 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8fb3d3-1dce-40c1-85b9-fea50213a32c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:34.534854 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fb8fb3d3-1dce-40c1-85b9-fea50213a32c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:34.664938 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:19:34.728872 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8fb3d3-1dce-40c1-85b9-fea50213a32c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:19:34.731106 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8fb3d3-1dce-40c1-85b9-fea50213a32c service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:19:34.761796 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 653/1325] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:19:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 596 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:19:37.833200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea160ef-7745-410e-b06e-5fd2a8279a7c None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:37.834264 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea160ef-7745-410e-b06e-5fd2a8279a7c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:37.834264 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea160ef-7745-410e-b06e-5fd2a8279a7c None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:37.835525 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 673/1326] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:19:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:19:37.841877 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28cce4b-3bc7-4ba8-b7bf-ca0826e2e1cf None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:37.842912 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28cce4b-3bc7-4ba8-b7bf-ca0826e2e1cf None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:37.843413 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28cce4b-3bc7-4ba8-b7bf-ca0826e2e1cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:37.925322 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-e28cce4b-3bc7-4ba8-b7bf-ca0826e2e1cf None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:19:37.926478 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e28cce4b-3bc7-4ba8-b7bf-ca0826e2e1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:38.052895 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 654/1327] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:19:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:19:38.097105 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea160ef-7745-410e-b06e-5fd2a8279a7c None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:19:38.099978 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4ea160ef-7745-410e-b06e-5fd2a8279a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:38.224005 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:19:38.308810 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea160ef-7745-410e-b06e-5fd2a8279a7c None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:19:38.476407 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535c83ad-a458-45f3-b38f-b02602a004c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:19:38.477497 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c83ad-a458-45f3-b38f-b02602a004c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:38.478041 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c83ad-a458-45f3-b38f-b02602a004c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:38.478457 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c83ad-a458-45f3-b38f-b02602a004c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:38.481421 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-535c83ad-a458-45f3-b38f-b02602a004c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:38.712924 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:19:38.717738 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535c83ad-a458-45f3-b38f-b02602a004c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:19:38.721337 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535c83ad-a458-45f3-b38f-b02602a004c4 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:19:38.762044 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 674/1328] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:19:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 666 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:19:41.493068 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a30ec2-6cbb-4e8c-bf2e-2b1727f34d9c None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:41.493681 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a30ec2-6cbb-4e8c-bf2e-2b1727f34d9c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:41.493681 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a30ec2-6cbb-4e8c-bf2e-2b1727f34d9c None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:41.495184 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 655/1329] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:19:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:19:41.502537 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8fc74-306a-4e83-be98-f877f458fddd None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:41.503429 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8fc74-306a-4e83-be98-f877f458fddd None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:41.503586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8fc74-306a-4e83-be98-f877f458fddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:41.590664 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-1ca8fc74-306a-4e83-be98-f877f458fddd None None] MFA Rules not processed for user `786bcdb4dc274deb8280030ff16c6d0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:19:41.591903 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-1ca8fc74-306a-4e83-be98-f877f458fddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:41.739052 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 675/1330] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:19:41 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:19:41.790978 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a30ec2-6cbb-4e8c-bf2e-2b1727f34d9c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:19:41.793790 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-46a30ec2-6cbb-4e8c-bf2e-2b1727f34d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:41.845965 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:19:41.981072 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a30ec2-6cbb-4e8c-bf2e-2b1727f34d9c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:19:42.137504 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4bdea8-dd85-43ee-9ac5-99b7f00e9514 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:19:42.138417 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4bdea8-dd85-43ee-9ac5-99b7f00e9514 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:42.138887 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4bdea8-dd85-43ee-9ac5-99b7f00e9514 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:42.139479 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4bdea8-dd85-43ee-9ac5-99b7f00e9514 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:42.142448 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2e4bdea8-dd85-43ee-9ac5-99b7f00e9514 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:42.275060 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:19:42.332306 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4bdea8-dd85-43ee-9ac5-99b7f00e9514 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:19:42.335126 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4bdea8-dd85-43ee-9ac5-99b7f00e9514 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:19:42.368921 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 656/1331] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:19:41 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:19:45.573032 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc693797-ff69-473a-ad05-d3577af1c2fb None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:45.573767 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc693797-ff69-473a-ad05-d3577af1c2fb None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:45.574110 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc693797-ff69-473a-ad05-d3577af1c2fb None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:45.576801 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 676/1332] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:19:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:19:45.585781 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cb739-8500-494e-9caa-814700a3c03b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:45.586357 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cb739-8500-494e-9caa-814700a3c03b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:45.586637 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cb739-8500-494e-9caa-814700a3c03b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:45.678287 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-e75cb739-8500-494e-9caa-814700a3c03b None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:19:45.679617 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e75cb739-8500-494e-9caa-814700a3c03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:45.806775 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 657/1333] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:19:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:19:45.849487 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc693797-ff69-473a-ad05-d3577af1c2fb None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:19:45.851328 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-cc693797-ff69-473a-ad05-d3577af1c2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:45.961081 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:19:46.039655 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc693797-ff69-473a-ad05-d3577af1c2fb None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:19:46.156634 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da420f52-8412-409c-a0e0-f04d3d0789f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:19:46.157812 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da420f52-8412-409c-a0e0-f04d3d0789f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:46.158340 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da420f52-8412-409c-a0e0-f04d3d0789f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:46.158818 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da420f52-8412-409c-a0e0-f04d3d0789f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:46.163104 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-da420f52-8412-409c-a0e0-f04d3d0789f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:46.377808 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da420f52-8412-409c-a0e0-f04d3d0789f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:19:46.380692 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da420f52-8412-409c-a0e0-f04d3d0789f8 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:19:46.417142 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:19:46.435292 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 677/1334] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:19:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:19:49.190160 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5a499-9127-4976-9cf1-904cd14be84b None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:49.190987 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5a499-9127-4976-9cf1-904cd14be84b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:49.190987 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5a499-9127-4976-9cf1-904cd14be84b None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:49.192371 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 658/1335] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:19:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:19:49.199400 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd6c60-3c2d-4098-b8b7-53f8e6601e51 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:49.200276 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd6c60-3c2d-4098-b8b7-53f8e6601e51 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:49.200276 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd6c60-3c2d-4098-b8b7-53f8e6601e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:49.283890 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-e3fd6c60-3c2d-4098-b8b7-53f8e6601e51 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:19:49.286076 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e3fd6c60-3c2d-4098-b8b7-53f8e6601e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:49.301763 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 678/1336] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:19:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:19:49.371616 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac5a499-9127-4976-9cf1-904cd14be84b None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:19:49.374268 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-bac5a499-9127-4976-9cf1-904cd14be84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:49.438925 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:19:49.519079 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac5a499-9127-4976-9cf1-904cd14be84b None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:19:49.524513 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce18b472-332a-44bf-ba76-3053ee0a254c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:19:49.525459 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18b472-332a-44bf-ba76-3053ee0a254c service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:49.525958 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18b472-332a-44bf-ba76-3053ee0a254c service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:49.526564 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18b472-332a-44bf-ba76-3053ee0a254c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:49.543787 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce18b472-332a-44bf-ba76-3053ee0a254c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:19:49.546486 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce18b472-332a-44bf-ba76-3053ee0a254c service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:19:49.565678 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 659/1337] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:19:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:19:52.443224 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d830c14-88e7-4c8d-859b-8da3c735f272 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:52.443822 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d830c14-88e7-4c8d-859b-8da3c735f272 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:52.443822 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d830c14-88e7-4c8d-859b-8da3c735f272 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:52.445578 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 679/1338] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:19:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:19:52.454407 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34effd9a-e64d-43d9-bf3c-cde48ff28eb1 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:52.455001 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34effd9a-e64d-43d9-bf3c-cde48ff28eb1 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:52.455001 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34effd9a-e64d-43d9-bf3c-cde48ff28eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:52.506307 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-34effd9a-e64d-43d9-bf3c-cde48ff28eb1 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:19:52.508179 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-34effd9a-e64d-43d9-bf3c-cde48ff28eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:52.623797 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 660/1339] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:19:52 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:19:52.663252 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d830c14-88e7-4c8d-859b-8da3c735f272 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:19:52.665366 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-0d830c14-88e7-4c8d-859b-8da3c735f272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:52.680854 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d830c14-88e7-4c8d-859b-8da3c735f272 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:19:52.685404 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51851b4-7d47-4c94-87c4-b6c16a13feaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:19:52.686119 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51851b4-7d47-4c94-87c4-b6c16a13feaf service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:52.686412 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51851b4-7d47-4c94-87c4-b6c16a13feaf service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:52.686697 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51851b4-7d47-4c94-87c4-b6c16a13feaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:52.698348 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51851b4-7d47-4c94-87c4-b6c16a13feaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:19:52.699837 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51851b4-7d47-4c94-87c4-b6c16a13feaf service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:19:52.711284 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 680/1340] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:19:52 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:19:57.921281 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82924331-7354-439c-b1b3-c199c67d16c6 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:57.921281 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82924331-7354-439c-b1b3-c199c67d16c6 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:57.921281 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82924331-7354-439c-b1b3-c199c67d16c6 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:57.922871 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 661/1341] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:19:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:19:57.929137 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b0b79-fd4f-4cc5-b659-160b4aa8ea6e None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:57.930018 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b0b79-fd4f-4cc5-b659-160b4aa8ea6e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:57.930018 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b0b79-fd4f-4cc5-b659-160b4aa8ea6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:57.950361 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-375b0b79-fd4f-4cc5-b659-160b4aa8ea6e None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:19:57.952488 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-375b0b79-fd4f-4cc5-b659-160b4aa8ea6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:57.961748 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 681/1342] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:19:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:19:58.019347 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82924331-7354-439c-b1b3-c199c67d16c6 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:19:58.038064 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82924331-7354-439c-b1b3-c199c67d16c6 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:19:58.049436 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ad0cab-6cce-41f3-95fa-295be393c2bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:19:58.050835 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad0cab-6cce-41f3-95fa-295be393c2bb service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:58.051385 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad0cab-6cce-41f3-95fa-295be393c2bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:58.051949 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad0cab-6cce-41f3-95fa-295be393c2bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:58.075943 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ad0cab-6cce-41f3-95fa-295be393c2bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:19:58.077580 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ad0cab-6cce-41f3-95fa-295be393c2bb service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:19:58.090441 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 662/1343] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:19:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:20:01.592686 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc432597-100f-4f12-92a0-176595ceb708 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:01.593254 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc432597-100f-4f12-92a0-176595ceb708 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:01.593891 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc432597-100f-4f12-92a0-176595ceb708 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:01.596575 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 682/1344] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:20:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:20:01.605722 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3caed31-3ca4-4e7b-a240-badf1dc883d8 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:01.606575 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3caed31-3ca4-4e7b-a240-badf1dc883d8 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:01.606737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3caed31-3ca4-4e7b-a240-badf1dc883d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:01.628775 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-e3caed31-3ca4-4e7b-a240-badf1dc883d8 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:20:01.630198 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e3caed31-3ca4-4e7b-a240-badf1dc883d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:20:01.640040 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 663/1345] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:20:01 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:20:01.679573 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc432597-100f-4f12-92a0-176595ceb708 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:20:01.691680 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc432597-100f-4f12-92a0-176595ceb708 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:20:01.696107 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55940641-5388-49bb-a2d0-7faba74f06bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:20:01.696951 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55940641-5388-49bb-a2d0-7faba74f06bb service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:01.697348 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55940641-5388-49bb-a2d0-7faba74f06bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:01.697800 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55940641-5388-49bb-a2d0-7faba74f06bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:01.709659 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55940641-5388-49bb-a2d0-7faba74f06bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:20:01.711397 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55940641-5388-49bb-a2d0-7faba74f06bb service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:20:01.727879 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 683/1346] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:20:01 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:20:07.626393 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ef421-3747-45d6-859d-995d6939ae53 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:07.626930 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ef421-3747-45d6-859d-995d6939ae53 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:07.627102 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ef421-3747-45d6-859d-995d6939ae53 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:07.629109 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 664/1347] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:20:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:20:07.635142 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f81434-485f-4709-861b-93aba11d4a44 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:07.636591 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f81434-485f-4709-861b-93aba11d4a44 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:07.636591 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f81434-485f-4709-861b-93aba11d4a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:07.657654 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-a1f81434-485f-4709-861b-93aba11d4a44 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:20:07.659073 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a1f81434-485f-4709-861b-93aba11d4a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:20:07.668571 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 684/1348] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:20:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:20:07.712701 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ef421-3747-45d6-859d-995d6939ae53 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:20:07.724487 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ef421-3747-45d6-859d-995d6939ae53 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:20:07.729025 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83018bad-b879-45e8-b47b-e6629128b2d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:20:07.729727 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83018bad-b879-45e8-b47b-e6629128b2d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:07.730014 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83018bad-b879-45e8-b47b-e6629128b2d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:07.730493 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83018bad-b879-45e8-b47b-e6629128b2d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:07.743152 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83018bad-b879-45e8-b47b-e6629128b2d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:20:07.746760 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83018bad-b879-45e8-b47b-e6629128b2d0 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:20:07.762395 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 665/1349] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:20:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:20:14.303308 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8892c1d5-9e6b-484f-8e37-33ca4def8698 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:14.303873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8892c1d5-9e6b-484f-8e37-33ca4def8698 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:14.304130 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8892c1d5-9e6b-484f-8e37-33ca4def8698 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:14.307529 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 685/1350] 10.222.0.87 () {54 vars in 921 bytes} [Tue Jun 21 18:20:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:20:14.316272 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad02be5-de2a-446f-9047-292485f10752 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:14.318071 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad02be5-de2a-446f-9047-292485f10752 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:14.318620 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad02be5-de2a-446f-9047-292485f10752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:14.489832 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-0ad02be5-de2a-446f-9047-292485f10752 None None] MFA Rules not processed for user `ef1d49b36cdc4665b82487be97e9dcd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:20:14.492450 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0ad02be5-de2a-446f-9047-292485f10752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:20:14.515070 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:20:14.657493 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 666/1351] 10.222.0.87 () {58 vars in 1018 bytes} [Tue Jun 21 18:20:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:20:17.658237 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=33 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:20:20.732361 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac614e-3b1f-44b7-b195-c3686cbde187 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:20.733420 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac614e-3b1f-44b7-b195-c3686cbde187 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:20.733588 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac614e-3b1f-44b7-b195-c3686cbde187 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:20.737218 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 686/1352] 10.222.0.87 () {54 vars in 921 bytes} [Tue Jun 21 18:20:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:20:20.746568 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057c072-d7b2-4ba8-a900-0a786e6aedab None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:20.747624 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057c072-d7b2-4ba8-a900-0a786e6aedab None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:20.748099 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057c072-d7b2-4ba8-a900-0a786e6aedab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:20.780791 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-5057c072-d7b2-4ba8-a900-0a786e6aedab None None] MFA Rules not processed for user `ef1d49b36cdc4665b82487be97e9dcd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:20:20.782925 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5057c072-d7b2-4ba8-a900-0a786e6aedab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:20:20.798393 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 667/1353] 10.222.0.87 () {58 vars in 1018 bytes} [Tue Jun 21 18:20:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:20:22.779911 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d1b7e-9b0b-43fc-a216-54238fecb415 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:22.780622 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d1b7e-9b0b-43fc-a216-54238fecb415 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:22.780622 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d1b7e-9b0b-43fc-a216-54238fecb415 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:22.781953 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 687/1354] 10.222.0.87 () {54 vars in 921 bytes} [Tue Jun 21 18:20:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:20:22.787906 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41098754-ab20-4bde-947a-182d3f777728 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:22.788505 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41098754-ab20-4bde-947a-182d3f777728 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:22.788591 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41098754-ab20-4bde-947a-182d3f777728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:22.811567 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-41098754-ab20-4bde-947a-182d3f777728 None None] MFA Rules not processed for user `ef1d49b36cdc4665b82487be97e9dcd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:20:22.813152 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-41098754-ab20-4bde-947a-182d3f777728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:20:22.824262 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 668/1355] 10.222.0.87 () {58 vars in 1018 bytes} [Tue Jun 21 18:20:22 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:20:27.171444 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df3537-3afb-41bb-ac22-47c89cb0b22a None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:27.171444 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df3537-3afb-41bb-ac22-47c89cb0b22a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:27.172343 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df3537-3afb-41bb-ac22-47c89cb0b22a None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:27.173509 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 688/1356] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:20:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:20:27.183331 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d1311-82cf-40f9-8a42-e0cc0ddc5b22 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:27.183908 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d1311-82cf-40f9-8a42-e0cc0ddc5b22 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:27.184351 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d1311-82cf-40f9-8a42-e0cc0ddc5b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:27.220075 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a16d1311-82cf-40f9-8a42-e0cc0ddc5b22 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:20:27.222295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a16d1311-82cf-40f9-8a42-e0cc0ddc5b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:20:27.236666 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 669/1357] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:20:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:20:27.340616 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2b47d-83c4-4d75-8cdc-0459dc9e5f8d None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:27.341437 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2b47d-83c4-4d75-8cdc-0459dc9e5f8d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:27.342212 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2b47d-83c4-4d75-8cdc-0459dc9e5f8d None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:27.345072 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 689/1358] 10.222.0.87 () {54 vars in 961 bytes} [Tue Jun 21 18:20:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:20:27.354636 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233ba82-5efd-4709-9030-53f42c3070ef None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:27.354999 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233ba82-5efd-4709-9030-53f42c3070ef None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:27.355610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233ba82-5efd-4709-9030-53f42c3070ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:27.468397 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a233ba82-5efd-4709-9030-53f42c3070ef None None] MFA Rules not processed for user `a1b3fa67905c4027b3656ac36e2b76c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:20:27.471536 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a233ba82-5efd-4709-9030-53f42c3070ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:20:27.607489 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 670/1359] 10.222.0.87 () {58 vars in 1058 bytes} [Tue Jun 21 18:20:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:20:27.616556 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a2b47d-83c4-4d75-8cdc-0459dc9e5f8d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:20:27.635558 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a2b47d-83c4-4d75-8cdc-0459dc9e5f8d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:20:27.642660 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7904894-d41f-4a1c-8cd9-e8c1766d29cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:20:27.643670 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7904894-d41f-4a1c-8cd9-e8c1766d29cf service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:27.644427 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7904894-d41f-4a1c-8cd9-e8c1766d29cf service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:27.644956 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7904894-d41f-4a1c-8cd9-e8c1766d29cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:27.657239 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7904894-d41f-4a1c-8cd9-e8c1766d29cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:20:27.659471 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7904894-d41f-4a1c-8cd9-e8c1766d29cf service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:20:27.673506 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 690/1360] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:20:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:20:30.479814 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78ec4d9-0745-437d-9589-a25747999fd3 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:30.481062 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78ec4d9-0745-437d-9589-a25747999fd3 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:30.481330 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78ec4d9-0745-437d-9589-a25747999fd3 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:30.483250 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 671/1361] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:20:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:20:30.489204 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027c074-28a1-4249-8f04-d105ffe29b6a None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:30.490070 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027c074-28a1-4249-8f04-d105ffe29b6a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:30.490070 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027c074-28a1-4249-8f04-d105ffe29b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:30.514758 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-c027c074-28a1-4249-8f04-d105ffe29b6a None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:20:30.516296 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c027c074-28a1-4249-8f04-d105ffe29b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:20:30.526666 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 691/1362] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:20:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:20:30.582871 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78ec4d9-0745-437d-9589-a25747999fd3 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:20:30.604523 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78ec4d9-0745-437d-9589-a25747999fd3 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:20:30.611233 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afedb83d-05ed-4eb8-9f74-1b020cb29b5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:20:30.612373 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afedb83d-05ed-4eb8-9f74-1b020cb29b5c service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:30.612872 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afedb83d-05ed-4eb8-9f74-1b020cb29b5c service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:30.613382 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afedb83d-05ed-4eb8-9f74-1b020cb29b5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:30.625135 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afedb83d-05ed-4eb8-9f74-1b020cb29b5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:20:30.627421 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afedb83d-05ed-4eb8-9f74-1b020cb29b5c service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:20:30.640481 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 672/1363] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:20:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:20:33.603055 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af569fd-f2a6-4e9d-8bc1-bc89302b7ebc None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:33.603761 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af569fd-f2a6-4e9d-8bc1-bc89302b7ebc None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:33.603761 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af569fd-f2a6-4e9d-8bc1-bc89302b7ebc None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:33.610102 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 692/1364] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:20:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:20:33.615551 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360eba42-745d-4760-946b-3d71e308b65b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:33.616622 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360eba42-745d-4760-946b-3d71e308b65b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:33.616876 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360eba42-745d-4760-946b-3d71e308b65b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:33.654249 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-360eba42-745d-4760-946b-3d71e308b65b None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:20:33.655755 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-360eba42-745d-4760-946b-3d71e308b65b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:20:33.669802 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 673/1365] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:20:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:20:33.755023 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb16bab-139f-4e29-85ef-40c3f5b4bc0c None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:33.755506 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb16bab-139f-4e29-85ef-40c3f5b4bc0c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:33.756230 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb16bab-139f-4e29-85ef-40c3f5b4bc0c None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:33.760314 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 693/1366] 10.222.0.87 () {54 vars in 961 bytes} [Tue Jun 21 18:20:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:20:33.771579 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5c9ee-2aa3-43de-8285-8b1e58cff564 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:33.772516 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5c9ee-2aa3-43de-8285-8b1e58cff564 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:33.773243 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5c9ee-2aa3-43de-8285-8b1e58cff564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:33.824570 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a6a5c9ee-2aa3-43de-8285-8b1e58cff564 None None] MFA Rules not processed for user `a1b3fa67905c4027b3656ac36e2b76c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:20:33.828488 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a6a5c9ee-2aa3-43de-8285-8b1e58cff564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:20:33.853076 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 674/1367] 10.222.0.87 () {58 vars in 1058 bytes} [Tue Jun 21 18:20:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:20:33.871294 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb16bab-139f-4e29-85ef-40c3f5b4bc0c None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:20:33.889593 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb16bab-139f-4e29-85ef-40c3f5b4bc0c None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:20:33.895838 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c0c54f-79e6-495f-b1c8-f92aa1decc03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:20:33.896602 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c0c54f-79e6-495f-b1c8-f92aa1decc03 service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:33.896891 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c0c54f-79e6-495f-b1c8-f92aa1decc03 service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:33.897205 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c0c54f-79e6-495f-b1c8-f92aa1decc03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:33.909170 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c0c54f-79e6-495f-b1c8-f92aa1decc03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:20:33.910872 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c0c54f-79e6-495f-b1c8-f92aa1decc03 service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:20:33.922826 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 694/1368] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:20:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:20:36.667683 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec29da-ada2-4700-9430-84b8b1a73d82 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:36.668641 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec29da-ada2-4700-9430-84b8b1a73d82 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:36.668641 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec29da-ada2-4700-9430-84b8b1a73d82 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:36.670063 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 675/1369] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:20:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:20:36.679537 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd044475-b642-4057-a1ee-093d24cb6e96 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:36.680316 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd044475-b642-4057-a1ee-093d24cb6e96 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:36.680866 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd044475-b642-4057-a1ee-093d24cb6e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:36.714684 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-bd044475-b642-4057-a1ee-093d24cb6e96 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:20:36.716733 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-bd044475-b642-4057-a1ee-093d24cb6e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:20:36.737496 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 695/1370] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:20:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:20:36.807983 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ec29da-ada2-4700-9430-84b8b1a73d82 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:20:36.827122 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ec29da-ada2-4700-9430-84b8b1a73d82 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:20:36.834358 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf63a50-9ee1-4acb-8217-5ef4757b42f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:20:36.835067 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf63a50-9ee1-4acb-8217-5ef4757b42f1 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:36.835392 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf63a50-9ee1-4acb-8217-5ef4757b42f1 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:36.835654 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf63a50-9ee1-4acb-8217-5ef4757b42f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:36.852515 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf63a50-9ee1-4acb-8217-5ef4757b42f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:20:36.855459 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf63a50-9ee1-4acb-8217-5ef4757b42f1 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:20:36.861445 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:20:36.871125 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 676/1371] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:20:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:20:37.558867 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce958f-ee89-48fc-95f4-f49a3b707146 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:37.559629 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce958f-ee89-48fc-95f4-f49a3b707146 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:37.560157 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce958f-ee89-48fc-95f4-f49a3b707146 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:37.561778 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 696/1372] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:20:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:20:37.568194 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99d765-463f-4199-857a-36f49753c6ec None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:37.568458 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99d765-463f-4199-857a-36f49753c6ec None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:37.568772 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99d765-463f-4199-857a-36f49753c6ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:37.589684 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-eb99d765-463f-4199-857a-36f49753c6ec None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:20:37.590983 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-eb99d765-463f-4199-857a-36f49753c6ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:20:37.601798 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 677/1373] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:20:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:20:37.923345 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a0e24-2974-4564-b5dd-e3f3589f9f92 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:37.923991 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a0e24-2974-4564-b5dd-e3f3589f9f92 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:37.923991 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a0e24-2974-4564-b5dd-e3f3589f9f92 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:37.925294 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 697/1374] 10.222.0.87 () {54 vars in 963 bytes} [Tue Jun 21 18:20:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:20:37.934584 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43be2316-816c-4ee3-994a-6bc19c7938d1 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:37.935381 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43be2316-816c-4ee3-994a-6bc19c7938d1 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:37.935381 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43be2316-816c-4ee3-994a-6bc19c7938d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:38.003724 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-43be2316-816c-4ee3-994a-6bc19c7938d1 None None] MFA Rules not processed for user `b683e6f78abd4986b415eb9ddf34cb8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:20:38.004993 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-43be2316-816c-4ee3-994a-6bc19c7938d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:20:38.083805 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 678/1375] 10.222.0.87 () {58 vars in 1060 bytes} [Tue Jun 21 18:20:37 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:20:38.093024 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a0e24-2974-4564-b5dd-e3f3589f9f92 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:20:38.107356 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a0e24-2974-4564-b5dd-e3f3589f9f92 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:20:38.116099 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266b4661-ab39-4345-82ae-a9740e0acf9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:20:38.117208 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b4661-ab39-4345-82ae-a9740e0acf9c service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:38.118046 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b4661-ab39-4345-82ae-a9740e0acf9c service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:38.118873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b4661-ab39-4345-82ae-a9740e0acf9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:38.137462 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266b4661-ab39-4345-82ae-a9740e0acf9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:20:38.140292 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266b4661-ab39-4345-82ae-a9740e0acf9c service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:20:38.160695 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 698/1376] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:20:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:20:39.350763 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b1b2a-a0e0-4538-927b-328cc81b4d61 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:39.351664 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b1b2a-a0e0-4538-927b-328cc81b4d61 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:39.351796 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b1b2a-a0e0-4538-927b-328cc81b4d61 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:39.354617 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 679/1377] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:20:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:20:39.363907 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e2e7d-c391-4437-9b49-10ea336eb8a9 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:39.364747 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e2e7d-c391-4437-9b49-10ea336eb8a9 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:39.364747 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e2e7d-c391-4437-9b49-10ea336eb8a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:39.394146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-620e2e7d-c391-4437-9b49-10ea336eb8a9 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:20:39.396209 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-620e2e7d-c391-4437-9b49-10ea336eb8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:20:39.417698 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 699/1378] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:20:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:20:39.499559 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380b1b2a-a0e0-4538-927b-328cc81b4d61 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:20:39.543682 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380b1b2a-a0e0-4538-927b-328cc81b4d61 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:20:39.559552 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58de214-0e15-4ed8-b339-07307b50c752 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:20:39.561091 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58de214-0e15-4ed8-b339-07307b50c752 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:39.562026 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58de214-0e15-4ed8-b339-07307b50c752 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:39.562627 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58de214-0e15-4ed8-b339-07307b50c752 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:39.600552 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58de214-0e15-4ed8-b339-07307b50c752 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:20:39.605148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58de214-0e15-4ed8-b339-07307b50c752 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:20:39.645465 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 680/1379] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:20:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:20:42.128017 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd57bf2-7268-4a4e-91a8-fed54855f80a None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:42.128612 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd57bf2-7268-4a4e-91a8-fed54855f80a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:42.128612 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd57bf2-7268-4a4e-91a8-fed54855f80a None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:42.130119 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 700/1380] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:20:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:20:42.138031 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2b7fc-49c8-44d2-85a4-d6787865da6b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:42.138734 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2b7fc-49c8-44d2-85a4-d6787865da6b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:42.138734 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2b7fc-49c8-44d2-85a4-d6787865da6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:42.162027 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-16f2b7fc-49c8-44d2-85a4-d6787865da6b None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:20:42.163395 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-16f2b7fc-49c8-44d2-85a4-d6787865da6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:20:42.173298 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 681/1381] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:20:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:20:42.226426 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd57bf2-7268-4a4e-91a8-fed54855f80a None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:20:42.245793 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd57bf2-7268-4a4e-91a8-fed54855f80a None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:20:42.253255 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c429ecab-a0b0-4c21-8c74-457561db23a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:20:42.254016 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c429ecab-a0b0-4c21-8c74-457561db23a8 service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:42.254308 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c429ecab-a0b0-4c21-8c74-457561db23a8 service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:42.254593 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c429ecab-a0b0-4c21-8c74-457561db23a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:42.265600 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c429ecab-a0b0-4c21-8c74-457561db23a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:20:42.267374 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c429ecab-a0b0-4c21-8c74-457561db23a8 service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:20:42.280436 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 701/1382] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:20:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:20:44.831521 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed814a93-144c-406a-9ccf-65122d11b061 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:44.832344 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed814a93-144c-406a-9ccf-65122d11b061 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:44.832344 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed814a93-144c-406a-9ccf-65122d11b061 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:44.833567 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 682/1383] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:20:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:20:44.839962 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a88fcbf-1f1d-4707-b5ee-828fd37126f3 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:44.840888 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a88fcbf-1f1d-4707-b5ee-828fd37126f3 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:44.841336 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a88fcbf-1f1d-4707-b5ee-828fd37126f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:44.867923 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-8a88fcbf-1f1d-4707-b5ee-828fd37126f3 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:20:44.869445 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8a88fcbf-1f1d-4707-b5ee-828fd37126f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:20:44.878947 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 702/1384] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:20:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:20:44.924230 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed814a93-144c-406a-9ccf-65122d11b061 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:20:44.944591 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed814a93-144c-406a-9ccf-65122d11b061 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:20:44.951686 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c589cb2-e70a-43d0-ac86-8015b97c2aaa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:20:44.952736 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c589cb2-e70a-43d0-ac86-8015b97c2aaa service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:44.953185 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c589cb2-e70a-43d0-ac86-8015b97c2aaa service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:44.953625 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c589cb2-e70a-43d0-ac86-8015b97c2aaa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:44.967265 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c589cb2-e70a-43d0-ac86-8015b97c2aaa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:20:44.968988 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c589cb2-e70a-43d0-ac86-8015b97c2aaa service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:20:44.982754 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 683/1385] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:20:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:20:47.884529 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de53d6-a658-4164-836e-0354421782fd None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:47.885808 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de53d6-a658-4164-836e-0354421782fd None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:47.886102 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de53d6-a658-4164-836e-0354421782fd None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:47.888335 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 703/1386] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:20:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:20:47.895444 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee42e5-63d6-4cc8-b241-f3bf758a28fd None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:47.896264 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee42e5-63d6-4cc8-b241-f3bf758a28fd None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:47.896411 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee42e5-63d6-4cc8-b241-f3bf758a28fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:47.928091 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-2fee42e5-63d6-4cc8-b241-f3bf758a28fd None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:20:47.930510 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2fee42e5-63d6-4cc8-b241-f3bf758a28fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:20:47.946111 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 684/1387] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:20:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:20:48.027793 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57de53d6-a658-4164-836e-0354421782fd None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:20:48.041722 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=37 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:20:48.054491 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57de53d6-a658-4164-836e-0354421782fd None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:20:48.064906 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe40d18-fec8-4c25-a5e5-ca129bb37117 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:20:48.066087 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe40d18-fec8-4c25-a5e5-ca129bb37117 service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:48.066591 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe40d18-fec8-4c25-a5e5-ca129bb37117 service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:48.067073 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe40d18-fec8-4c25-a5e5-ca129bb37117 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:48.086200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe40d18-fec8-4c25-a5e5-ca129bb37117 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:20:48.089890 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe40d18-fec8-4c25-a5e5-ca129bb37117 service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:20:48.112126 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 704/1388] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:20:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:20:50.672192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb8ebd7-0e47-41eb-9b52-438b2bb79df1 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:50.672802 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb8ebd7-0e47-41eb-9b52-438b2bb79df1 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:50.672802 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb8ebd7-0e47-41eb-9b52-438b2bb79df1 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:50.674289 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 685/1389] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:20:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:20:50.682001 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d595900-1478-4e65-a0b0-d655d9a301bf None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:50.683190 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d595900-1478-4e65-a0b0-d655d9a301bf None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:50.683190 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d595900-1478-4e65-a0b0-d655d9a301bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:50.716903 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-4d595900-1478-4e65-a0b0-d655d9a301bf None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:20:50.719345 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4d595900-1478-4e65-a0b0-d655d9a301bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:20:50.734960 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 705/1390] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:20:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:20:50.795901 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb8ebd7-0e47-41eb-9b52-438b2bb79df1 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:20:50.821824 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb8ebd7-0e47-41eb-9b52-438b2bb79df1 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:20:50.830714 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6749802c-eeed-4421-a614-a849ff9d3f5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:20:50.831778 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749802c-eeed-4421-a614-a849ff9d3f5c service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:50.832156 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749802c-eeed-4421-a614-a849ff9d3f5c service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:50.832603 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749802c-eeed-4421-a614-a849ff9d3f5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:50.847934 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6749802c-eeed-4421-a614-a849ff9d3f5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:20:50.854588 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6749802c-eeed-4421-a614-a849ff9d3f5c service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:20:50.870685 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 686/1391] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:20:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:20:53.659604 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef2d45-ee62-4872-9430-1eb2559bf6a5 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:53.660502 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef2d45-ee62-4872-9430-1eb2559bf6a5 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:53.660973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef2d45-ee62-4872-9430-1eb2559bf6a5 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:53.663915 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 706/1392] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:20:53 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:20:53.673290 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24379d33-df34-41a0-a39b-af1ad7e8737b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:53.674224 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24379d33-df34-41a0-a39b-af1ad7e8737b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:53.674649 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24379d33-df34-41a0-a39b-af1ad7e8737b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:53.707915 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-24379d33-df34-41a0-a39b-af1ad7e8737b None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:20:53.709676 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-24379d33-df34-41a0-a39b-af1ad7e8737b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:20:53.722839 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 687/1393] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:20:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:20:53.779587 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ef2d45-ee62-4872-9430-1eb2559bf6a5 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:20:53.801765 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ef2d45-ee62-4872-9430-1eb2559bf6a5 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:20:53.812081 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3417d2bc-303e-4b40-8e3a-c964667fa50d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:20:53.813620 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417d2bc-303e-4b40-8e3a-c964667fa50d service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:53.814468 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417d2bc-303e-4b40-8e3a-c964667fa50d service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:53.814991 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417d2bc-303e-4b40-8e3a-c964667fa50d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:53.833413 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3417d2bc-303e-4b40-8e3a-c964667fa50d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:20:53.836527 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3417d2bc-303e-4b40-8e3a-c964667fa50d service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:20:53.856809 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 707/1394] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:20:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:20:56.638294 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae533a4-65c1-4fd4-9e88-5dc4d1bdfcb2 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:56.638994 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae533a4-65c1-4fd4-9e88-5dc4d1bdfcb2 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:56.638994 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae533a4-65c1-4fd4-9e88-5dc4d1bdfcb2 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:56.640553 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 688/1395] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:20:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:20:56.646354 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338cd2ef-22df-4f7d-b430-79747091f679 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:56.647018 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338cd2ef-22df-4f7d-b430-79747091f679 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:56.647018 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338cd2ef-22df-4f7d-b430-79747091f679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:56.667905 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-338cd2ef-22df-4f7d-b430-79747091f679 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:20:56.669186 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-338cd2ef-22df-4f7d-b430-79747091f679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:20:56.678322 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 708/1396] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:20:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:20:56.738505 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae533a4-65c1-4fd4-9e88-5dc4d1bdfcb2 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:20:56.756962 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae533a4-65c1-4fd4-9e88-5dc4d1bdfcb2 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:20:56.763380 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd4cc59-eba4-4f3f-b791-97db3b8f76f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:20:56.764111 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd4cc59-eba4-4f3f-b791-97db3b8f76f4 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:56.764390 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd4cc59-eba4-4f3f-b791-97db3b8f76f4 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:56.764667 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd4cc59-eba4-4f3f-b791-97db3b8f76f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:56.774622 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd4cc59-eba4-4f3f-b791-97db3b8f76f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:20:56.776307 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd4cc59-eba4-4f3f-b791-97db3b8f76f4 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:20:56.787833 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 689/1397] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:20:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:20:57.396091 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a7ff71-c421-4381-93fb-b48a7e175135 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:57.396482 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a7ff71-c421-4381-93fb-b48a7e175135 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:57.396827 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a7ff71-c421-4381-93fb-b48a7e175135 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:57.398404 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 709/1398] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:20:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:20:57.405469 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c1822-6eca-4a4e-afc8-79263b8c2c50 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:57.405971 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c1822-6eca-4a4e-afc8-79263b8c2c50 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:57.406379 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c1822-6eca-4a4e-afc8-79263b8c2c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:57.427853 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b94c1822-6eca-4a4e-afc8-79263b8c2c50 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:20:57.429060 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b94c1822-6eca-4a4e-afc8-79263b8c2c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:20:57.439022 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 690/1399] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:20:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:20:57.531566 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076caa9-c723-4015-81f2-f23c841cde32 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:57.532166 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076caa9-c723-4015-81f2-f23c841cde32 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:57.532641 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076caa9-c723-4015-81f2-f23c841cde32 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:57.536057 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 710/1400] 10.222.0.87 () {54 vars in 962 bytes} [Tue Jun 21 18:20:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:20:57.544511 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea33b03f-5e5b-4981-8d13-9679f1efe951 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:57.544814 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea33b03f-5e5b-4981-8d13-9679f1efe951 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:57.545288 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea33b03f-5e5b-4981-8d13-9679f1efe951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:57.649939 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-ea33b03f-5e5b-4981-8d13-9679f1efe951 None None] MFA Rules not processed for user `e7da4bf6f3be47b4aa25c6240cb4be03`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:20:57.652186 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ea33b03f-5e5b-4981-8d13-9679f1efe951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:20:57.689470 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:20:57.764441 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 691/1401] 10.222.0.87 () {58 vars in 1059 bytes} [Tue Jun 21 18:20:57 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:20:57.777781 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3076caa9-c723-4015-81f2-f23c841cde32 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:20:57.798647 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3076caa9-c723-4015-81f2-f23c841cde32 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:20:57.805628 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3acac3-4c8f-45df-a968-dee0fb54135a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7da4bf6f3be47b4aa25c6240cb4be03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:20:57.806805 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3acac3-4c8f-45df-a968-dee0fb54135a service glance] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:57.807683 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3acac3-4c8f-45df-a968-dee0fb54135a service glance] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:57.808191 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3acac3-4c8f-45df-a968-dee0fb54135a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:57.827222 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3acac3-4c8f-45df-a968-dee0fb54135a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:20:57.829344 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3acac3-4c8f-45df-a968-dee0fb54135a service glance] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:20:57.847461 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 711/1402] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:20:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:20:58.811603 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd800bcb-a667-4c04-86a1-8fe2179f807b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:58.812430 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd800bcb-a667-4c04-86a1-8fe2179f807b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:58.812430 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd800bcb-a667-4c04-86a1-8fe2179f807b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:58.900335 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-cd800bcb-a667-4c04-86a1-8fe2179f807b None None] MFA Rules not processed for user `09c9fa7dccb8491c9ad3b43967345ea2`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:20:58.901789 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-cd800bcb-a667-4c04-86a1-8fe2179f807b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:20:59.027209 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 692/1403] 10.222.0.87 () {58 vars in 1009 bytes} [Tue Jun 21 18:20:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:20:59.100174 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af3ca0-8200-4ec1-b60c-031a4cebd6eb None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:59.101007 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af3ca0-8200-4ec1-b60c-031a4cebd6eb None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:59.101145 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af3ca0-8200-4ec1-b60c-031a4cebd6eb None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:59.104047 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 712/1404] 10.222.0.87 () {54 vars in 942 bytes} [Tue Jun 21 18:20:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:20:59.115311 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11614ff6-1e1c-47be-8db0-e66856b69f66 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:59.115799 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11614ff6-1e1c-47be-8db0-e66856b69f66 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:59.116295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11614ff6-1e1c-47be-8db0-e66856b69f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:59.188673 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-11614ff6-1e1c-47be-8db0-e66856b69f66 None None] MFA Rules not processed for user `515b3571045e4fe29565307c8ecc10b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:20:59.189630 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-11614ff6-1e1c-47be-8db0-e66856b69f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:20:59.291077 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af82668-665b-48ee-a48d-c4eb4223a9b7 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:59.291415 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af82668-665b-48ee-a48d-c4eb4223a9b7 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:59.291859 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af82668-665b-48ee-a48d-c4eb4223a9b7 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:59.293547 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 713/1405] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:20:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:20:59.300214 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ce988-27bf-41f0-804b-b9ca115e3dfa None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:59.300654 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ce988-27bf-41f0-804b-b9ca115e3dfa None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:59.301068 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ce988-27bf-41f0-804b-b9ca115e3dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:59.321329 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-726ce988-27bf-41f0-804b-b9ca115e3dfa None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:20:59.323013 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-726ce988-27bf-41f0-804b-b9ca115e3dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:20:59.323223 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:20:59.336099 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 714/1406] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:20:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:20:59.336694 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 693/1407] 10.222.0.87 () {58 vars in 1039 bytes} [Tue Jun 21 18:20:59 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:20:59.345950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726ce988-27bf-41f0-804b-b9ca115e3dfa None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:20:59.360378 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726ce988-27bf-41f0-804b-b9ca115e3dfa None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:20:59.367095 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4484dfed-a32a-48af-85ed-e0e9b847e037 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '515b3571045e4fe29565307c8ecc10b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:20:59.367574 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484dfed-a32a-48af-85ed-e0e9b847e037 service swift] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:59.367897 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484dfed-a32a-48af-85ed-e0e9b847e037 service swift] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:59.368174 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484dfed-a32a-48af-85ed-e0e9b847e037 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:59.378085 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4484dfed-a32a-48af-85ed-e0e9b847e037 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:20:59.379759 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4484dfed-a32a-48af-85ed-e0e9b847e037 service swift] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:20:59.386697 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11614ff6-1e1c-47be-8db0-e66856b69f66 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:20:59.390247 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 715/1408] 10.222.0.87 () {60 vars in 1384 bytes} [Tue Jun 21 18:20:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 21 18:20:59.403762 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11614ff6-1e1c-47be-8db0-e66856b69f66 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:20:59.413000 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d4715-102d-4b7e-ac6e-5b81e0c39677 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:20:59.414207 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d4715-102d-4b7e-ac6e-5b81e0c39677 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:59.414982 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d4715-102d-4b7e-ac6e-5b81e0c39677 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:59.415868 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d4715-102d-4b7e-ac6e-5b81e0c39677 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:59.429619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752d4715-102d-4b7e-ac6e-5b81e0c39677 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:20:59.433420 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752d4715-102d-4b7e-ac6e-5b81e0c39677 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:20:59.450751 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 694/1409] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:20:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:21:09.386224 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=32 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:21:09.444763 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=6 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:21:27.293375 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4956e-0801-4dd1-be2f-135d8b76691a None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:27.294321 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4956e-0801-4dd1-be2f-135d8b76691a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:27.294321 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4956e-0801-4dd1-be2f-135d8b76691a None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:27.295435 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 716/1410] 10.222.0.87 () {54 vars in 918 bytes} [Tue Jun 21 18:21:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:21:27.301783 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d599f-517f-4a8f-baa5-5cc74e8391fd None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:27.302377 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d599f-517f-4a8f-baa5-5cc74e8391fd None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:27.302554 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d599f-517f-4a8f-baa5-5cc74e8391fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:27.330416 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-1a2d599f-517f-4a8f-baa5-5cc74e8391fd None None] MFA Rules not processed for user `ef1d49b36cdc4665b82487be97e9dcd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:21:27.332435 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1a2d599f-517f-4a8f-baa5-5cc74e8391fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:21:27.348272 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 695/1411] 10.222.0.87 () {58 vars in 1015 bytes} [Tue Jun 21 18:21:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:21:27.364895 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d97b37-bc64-4e16-9c2c-bef26d360397 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:27.365202 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d97b37-bc64-4e16-9c2c-bef26d360397 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:27.365437 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d97b37-bc64-4e16-9c2c-bef26d360397 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:27.367495 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 717/1412] 10.222.0.87 () {54 vars in 918 bytes} [Tue Jun 21 18:21:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:21:27.374970 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92204f1c-3786-432a-bf5e-ac54549deb2b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:27.375409 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92204f1c-3786-432a-bf5e-ac54549deb2b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:27.375806 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92204f1c-3786-432a-bf5e-ac54549deb2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:27.398119 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-92204f1c-3786-432a-bf5e-ac54549deb2b None None] MFA Rules not processed for user `a1b3fa67905c4027b3656ac36e2b76c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:21:27.399379 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-92204f1c-3786-432a-bf5e-ac54549deb2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:21:27.415289 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 696/1413] 10.222.0.87 () {58 vars in 1015 bytes} [Tue Jun 21 18:21:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:21:30.199423 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050a9ef-6641-42f5-b38e-7cd3492bd94d None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:30.200036 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050a9ef-6641-42f5-b38e-7cd3492bd94d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:30.200036 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050a9ef-6641-42f5-b38e-7cd3492bd94d None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:30.201388 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 718/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 18:21:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:21:30.207539 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1b8253-d119-4318-9af8-e7245c7610e0 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:30.208176 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1b8253-d119-4318-9af8-e7245c7610e0 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:30.208176 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1b8253-d119-4318-9af8-e7245c7610e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:30.237250 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-8d1b8253-d119-4318-9af8-e7245c7610e0 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:21:30.239712 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8d1b8253-d119-4318-9af8-e7245c7610e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:21:30.254725 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 697/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 18:21:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:21:30.314419 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f050a9ef-6641-42f5-b38e-7cd3492bd94d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:21:30.331785 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f050a9ef-6641-42f5-b38e-7cd3492bd94d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:21:30.338642 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d02c1e-57cb-4cec-a28a-5e10b7d1d5e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:21:30.339661 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d02c1e-57cb-4cec-a28a-5e10b7d1d5e6 service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:30.340043 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d02c1e-57cb-4cec-a28a-5e10b7d1d5e6 service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:30.340411 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d02c1e-57cb-4cec-a28a-5e10b7d1d5e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:30.351114 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d02c1e-57cb-4cec-a28a-5e10b7d1d5e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:21:30.352869 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d02c1e-57cb-4cec-a28a-5e10b7d1d5e6 service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:21:30.365797 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 719/1416] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:21:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:21:31.397566 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d75d59-0758-4295-a5fe-87042c5e30a3 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:31.398450 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d75d59-0758-4295-a5fe-87042c5e30a3 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:31.398450 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d75d59-0758-4295-a5fe-87042c5e30a3 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:31.399986 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 698/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 18:21:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:21:31.408523 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c544599-1159-4d8c-ac48-db8b7722cbfb None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:31.409176 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c544599-1159-4d8c-ac48-db8b7722cbfb None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:31.409360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c544599-1159-4d8c-ac48-db8b7722cbfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:31.441784 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-9c544599-1159-4d8c-ac48-db8b7722cbfb None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:21:31.444119 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-9c544599-1159-4d8c-ac48-db8b7722cbfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:21:31.459394 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 720/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 18:21:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:21:31.619178 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d75d59-0758-4295-a5fe-87042c5e30a3 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:21:31.634968 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d75d59-0758-4295-a5fe-87042c5e30a3 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:21:31.641237 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9001e-b71f-424c-9bb8-5ce325eb2508 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:21:31.642005 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9001e-b71f-424c-9bb8-5ce325eb2508 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:31.642291 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9001e-b71f-424c-9bb8-5ce325eb2508 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:31.642570 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9001e-b71f-424c-9bb8-5ce325eb2508 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:31.652547 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a9001e-b71f-424c-9bb8-5ce325eb2508 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:21:31.654192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a9001e-b71f-424c-9bb8-5ce325eb2508 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:21:31.666387 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 699/1419] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:21:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:21:32.736829 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2889ce-74a4-46bc-b4af-3db2c0606254 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:32.737515 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2889ce-74a4-46bc-b4af-3db2c0606254 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:32.737798 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2889ce-74a4-46bc-b4af-3db2c0606254 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:32.740818 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 721/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 18:21:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:21:32.750623 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0978ff9f-37ae-47ab-9262-410feef881d4 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:32.751504 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0978ff9f-37ae-47ab-9262-410feef881d4 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:32.751693 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0978ff9f-37ae-47ab-9262-410feef881d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:32.781118 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-0978ff9f-37ae-47ab-9262-410feef881d4 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:21:32.782617 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0978ff9f-37ae-47ab-9262-410feef881d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:21:32.793378 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 700/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 18:21:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:21:33.000721 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2889ce-74a4-46bc-b4af-3db2c0606254 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:21:33.019212 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2889ce-74a4-46bc-b4af-3db2c0606254 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:21:33.026246 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c029e5-d639-4ff3-8022-51e138193f67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:21:33.026950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c029e5-d639-4ff3-8022-51e138193f67 service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:33.027245 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c029e5-d639-4ff3-8022-51e138193f67 service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:33.027626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c029e5-d639-4ff3-8022-51e138193f67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:33.038590 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c029e5-d639-4ff3-8022-51e138193f67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:21:33.040633 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c029e5-d639-4ff3-8022-51e138193f67 service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:21:33.053275 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 722/1422] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:21:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:21:34.055904 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d835e1c-7295-4837-a2c2-94f8125e14b0 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:34.056749 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d835e1c-7295-4837-a2c2-94f8125e14b0 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:34.056972 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d835e1c-7295-4837-a2c2-94f8125e14b0 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:34.059034 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 701/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 18:21:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:21:34.066473 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e146ca-d18d-4c8b-95da-f1258408eed1 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:34.067357 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e146ca-d18d-4c8b-95da-f1258408eed1 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:34.067357 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e146ca-d18d-4c8b-95da-f1258408eed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:34.089202 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-08e146ca-d18d-4c8b-95da-f1258408eed1 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:21:34.090527 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-08e146ca-d18d-4c8b-95da-f1258408eed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:21:34.100807 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 723/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 18:21:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:21:34.259602 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d835e1c-7295-4837-a2c2-94f8125e14b0 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:21:34.286352 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d835e1c-7295-4837-a2c2-94f8125e14b0 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:21:34.296508 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2e323d-1655-4a73-bfcf-149d7b5523a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:21:34.297706 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2e323d-1655-4a73-bfcf-149d7b5523a8 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:34.298213 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2e323d-1655-4a73-bfcf-149d7b5523a8 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:34.298737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2e323d-1655-4a73-bfcf-149d7b5523a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:34.316124 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2e323d-1655-4a73-bfcf-149d7b5523a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:21:34.319605 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2e323d-1655-4a73-bfcf-149d7b5523a8 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:21:34.333241 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 702/1425] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:21:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:21:35.522272 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26d287-5e24-415e-ac55-d3b152e67093 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:35.523167 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26d287-5e24-415e-ac55-d3b152e67093 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:35.523167 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26d287-5e24-415e-ac55-d3b152e67093 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:35.524541 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 724/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 18:21:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:21:35.530728 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01cc81d-3efb-486d-9cef-2fe647cb76a3 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:35.531691 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01cc81d-3efb-486d-9cef-2fe647cb76a3 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:35.531691 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01cc81d-3efb-486d-9cef-2fe647cb76a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:35.552300 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-c01cc81d-3efb-486d-9cef-2fe647cb76a3 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:21:35.553867 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c01cc81d-3efb-486d-9cef-2fe647cb76a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:21:35.562925 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 703/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 18:21:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:21:35.723934 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea26d287-5e24-415e-ac55-d3b152e67093 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:21:35.739878 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea26d287-5e24-415e-ac55-d3b152e67093 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:21:35.746191 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc66f19-0c6b-4537-ba40-aeee9e3c77b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:21:35.746875 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc66f19-0c6b-4537-ba40-aeee9e3c77b0 service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:35.747156 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc66f19-0c6b-4537-ba40-aeee9e3c77b0 service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:35.747459 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc66f19-0c6b-4537-ba40-aeee9e3c77b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:35.757252 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc66f19-0c6b-4537-ba40-aeee9e3c77b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:21:35.758939 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc66f19-0c6b-4537-ba40-aeee9e3c77b0 service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:21:35.770443 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 725/1428] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:21:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:21:36.663037 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be19a832-3868-4277-8844-5c34f5a85f5d None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:36.663985 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be19a832-3868-4277-8844-5c34f5a85f5d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:36.664156 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be19a832-3868-4277-8844-5c34f5a85f5d None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:36.665957 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 704/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 18:21:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:21:36.672226 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bee02-3905-4908-93c1-ced9da1cd444 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:36.672966 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bee02-3905-4908-93c1-ced9da1cd444 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:36.672966 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bee02-3905-4908-93c1-ced9da1cd444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:36.693461 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-966bee02-3905-4908-93c1-ced9da1cd444 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:21:36.694859 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-966bee02-3905-4908-93c1-ced9da1cd444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:21:36.703974 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 726/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 18:21:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:21:36.860914 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be19a832-3868-4277-8844-5c34f5a85f5d None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:21:36.880106 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be19a832-3868-4277-8844-5c34f5a85f5d None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:21:36.886859 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59ac0eb-86f6-41e4-bbba-09c2162cf45e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:21:36.887586 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ac0eb-86f6-41e4-bbba-09c2162cf45e service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:36.887874 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ac0eb-86f6-41e4-bbba-09c2162cf45e service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:36.888150 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ac0eb-86f6-41e4-bbba-09c2162cf45e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:36.899019 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59ac0eb-86f6-41e4-bbba-09c2162cf45e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:21:36.900813 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59ac0eb-86f6-41e4-bbba-09c2162cf45e service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:21:36.913116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 705/1431] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:21:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:21:37.850936 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71583e7-b9b4-4098-a6f1-656c33518683 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:37.851757 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71583e7-b9b4-4098-a6f1-656c33518683 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:37.851966 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71583e7-b9b4-4098-a6f1-656c33518683 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:37.854591 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 727/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 18:21:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:21:37.863980 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c7ced-cb9d-4daf-be49-9eff0fc1160c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:37.864592 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c7ced-cb9d-4daf-be49-9eff0fc1160c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:37.864918 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c7ced-cb9d-4daf-be49-9eff0fc1160c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:37.895024 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-0f8c7ced-cb9d-4daf-be49-9eff0fc1160c None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:21:37.896277 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0f8c7ced-cb9d-4daf-be49-9eff0fc1160c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:21:37.913528 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 706/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 18:21:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:21:38.224410 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71583e7-b9b4-4098-a6f1-656c33518683 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:21:38.246312 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71583e7-b9b4-4098-a6f1-656c33518683 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:21:38.253193 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d98d32b-8494-4523-bfa8-a4defe45e108 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:21:38.253998 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98d32b-8494-4523-bfa8-a4defe45e108 service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:38.254321 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98d32b-8494-4523-bfa8-a4defe45e108 service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:38.254631 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98d32b-8494-4523-bfa8-a4defe45e108 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:38.268132 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d98d32b-8494-4523-bfa8-a4defe45e108 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:21:38.271327 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d98d32b-8494-4523-bfa8-a4defe45e108 service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:21:38.291516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 728/1434] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:21:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:21:39.302083 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2e59f-9e8f-461e-be8e-0b0a86abb9ce None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:39.302985 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2e59f-9e8f-461e-be8e-0b0a86abb9ce None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:39.303158 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2e59f-9e8f-461e-be8e-0b0a86abb9ce None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:39.305744 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 707/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 18:21:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:21:39.313147 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8e344-fbb5-437f-9fb6-4094f6de7087 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:39.314034 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8e344-fbb5-437f-9fb6-4094f6de7087 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:39.314272 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8e344-fbb5-437f-9fb6-4094f6de7087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:39.340880 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-a8c8e344-fbb5-437f-9fb6-4094f6de7087 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:21:39.342326 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a8c8e344-fbb5-437f-9fb6-4094f6de7087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:21:39.351687 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 729/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 18:21:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:21:39.514681 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f2e59f-9e8f-461e-be8e-0b0a86abb9ce None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:21:39.531919 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f2e59f-9e8f-461e-be8e-0b0a86abb9ce None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:21:39.538907 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdfc4f5-76c2-4240-bef0-f62bbc979981 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:21:39.539769 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdfc4f5-76c2-4240-bef0-f62bbc979981 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:39.540152 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdfc4f5-76c2-4240-bef0-f62bbc979981 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:39.540529 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdfc4f5-76c2-4240-bef0-f62bbc979981 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:39.551398 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdfc4f5-76c2-4240-bef0-f62bbc979981 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:21:39.553346 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdfc4f5-76c2-4240-bef0-f62bbc979981 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:21:39.566008 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 708/1437] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:21:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:21:40.485583 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa32368-089c-423f-8569-9e451fdd61d9 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:40.486417 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa32368-089c-423f-8569-9e451fdd61d9 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:40.486417 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa32368-089c-423f-8569-9e451fdd61d9 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:40.488106 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 730/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 18:21:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:21:40.495269 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee68e70-8f44-4006-8627-ac5069b44f62 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:40.496325 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee68e70-8f44-4006-8627-ac5069b44f62 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:40.496325 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee68e70-8f44-4006-8627-ac5069b44f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:40.517387 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-dee68e70-8f44-4006-8627-ac5069b44f62 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:21:40.518779 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-dee68e70-8f44-4006-8627-ac5069b44f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:21:40.529114 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 709/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 18:21:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:21:40.703773 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa32368-089c-423f-8569-9e451fdd61d9 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:21:40.728660 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa32368-089c-423f-8569-9e451fdd61d9 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:21:40.736740 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3611a645-160b-4fd5-9e33-a8b542153249 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:21:40.737297 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611a645-160b-4fd5-9e33-a8b542153249 service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:40.737579 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611a645-160b-4fd5-9e33-a8b542153249 service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:40.738197 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611a645-160b-4fd5-9e33-a8b542153249 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:40.748983 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3611a645-160b-4fd5-9e33-a8b542153249 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:21:40.751356 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3611a645-160b-4fd5-9e33-a8b542153249 service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:21:40.763140 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 731/1440] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:21:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:21:43.037221 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bddce4b-8877-4c90-bae2-510bb8bd63c5 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:43.037871 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bddce4b-8877-4c90-bae2-510bb8bd63c5 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:43.037995 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bddce4b-8877-4c90-bae2-510bb8bd63c5 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:43.039396 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 710/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 18:21:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:21:43.045663 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15a6ce-df5c-4334-99c4-cbb1d1b2ecf2 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:43.046230 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15a6ce-df5c-4334-99c4-cbb1d1b2ecf2 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:43.046338 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15a6ce-df5c-4334-99c4-cbb1d1b2ecf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:43.068211 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-ac15a6ce-df5c-4334-99c4-cbb1d1b2ecf2 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:21:43.069463 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ac15a6ce-df5c-4334-99c4-cbb1d1b2ecf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:21:43.078851 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 732/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 18:21:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:21:43.122742 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bddce4b-8877-4c90-bae2-510bb8bd63c5 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:21:43.136816 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bddce4b-8877-4c90-bae2-510bb8bd63c5 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:21:43.141288 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a547c5-0866-4e05-a438-515be9eb526b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:21:43.142031 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a547c5-0866-4e05-a438-515be9eb526b service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:43.142322 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a547c5-0866-4e05-a438-515be9eb526b service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:43.142602 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a547c5-0866-4e05-a438-515be9eb526b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:43.153910 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a547c5-0866-4e05-a438-515be9eb526b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:21:43.155400 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a547c5-0866-4e05-a438-515be9eb526b service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:21:43.167370 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 711/1443] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:21:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:21:45.645673 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e995749-0ad9-4e58-af99-63d3a2fa4460 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:45.646694 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e995749-0ad9-4e58-af99-63d3a2fa4460 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:45.646694 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e995749-0ad9-4e58-af99-63d3a2fa4460 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:45.648577 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 733/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 18:21:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:21:45.655642 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ca12f-55a1-4445-a964-9118e79a8c9d None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:45.656383 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ca12f-55a1-4445-a964-9118e79a8c9d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:45.656383 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ca12f-55a1-4445-a964-9118e79a8c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:45.677038 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-410ca12f-55a1-4445-a964-9118e79a8c9d None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:21:45.678457 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-410ca12f-55a1-4445-a964-9118e79a8c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:21:45.687041 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 712/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 18:21:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:21:45.754362 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e995749-0ad9-4e58-af99-63d3a2fa4460 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:21:45.774916 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e995749-0ad9-4e58-af99-63d3a2fa4460 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:21:45.780990 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899699de-663c-41dd-ae76-98c60581e308 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:21:45.782025 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899699de-663c-41dd-ae76-98c60581e308 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:45.782530 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899699de-663c-41dd-ae76-98c60581e308 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:45.782969 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899699de-663c-41dd-ae76-98c60581e308 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:45.799799 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899699de-663c-41dd-ae76-98c60581e308 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:21:45.802487 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899699de-663c-41dd-ae76-98c60581e308 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:21:45.818804 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 734/1446] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:21:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:21:48.700068 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d83a35-953d-454a-add8-0573ff59f3a2 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:48.700942 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d83a35-953d-454a-add8-0573ff59f3a2 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:48.700942 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d83a35-953d-454a-add8-0573ff59f3a2 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:48.702322 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 713/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 18:21:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:21:48.709432 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee87ac6-ee26-4e37-aaf0-f3adad50c6fc None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:48.710298 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee87ac6-ee26-4e37-aaf0-f3adad50c6fc None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:48.710503 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee87ac6-ee26-4e37-aaf0-f3adad50c6fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:48.732192 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-0ee87ac6-ee26-4e37-aaf0-f3adad50c6fc None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:21:48.733522 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-0ee87ac6-ee26-4e37-aaf0-f3adad50c6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:21:48.742771 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 735/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 18:21:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:21:48.788927 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d83a35-953d-454a-add8-0573ff59f3a2 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:21:48.805299 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d83a35-953d-454a-add8-0573ff59f3a2 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:21:48.811012 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496294e7-862e-448d-a47d-ce49b20de036 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:21:48.811954 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496294e7-862e-448d-a47d-ce49b20de036 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:48.812364 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496294e7-862e-448d-a47d-ce49b20de036 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:48.812773 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496294e7-862e-448d-a47d-ce49b20de036 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:48.827944 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496294e7-862e-448d-a47d-ce49b20de036 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:21:48.830477 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496294e7-862e-448d-a47d-ce49b20de036 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:21:48.846135 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 714/1449] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:21:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:21:51.582410 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e294b4-f5ea-479a-b8a4-85f4008faddc None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:51.583003 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e294b4-f5ea-479a-b8a4-85f4008faddc None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:51.583003 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e294b4-f5ea-479a-b8a4-85f4008faddc None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:51.585178 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 736/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 18:21:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:21:51.591843 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267bebb-e197-4b2b-acc8-c8a3a02ec65b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:51.592536 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267bebb-e197-4b2b-acc8-c8a3a02ec65b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:51.592536 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267bebb-e197-4b2b-acc8-c8a3a02ec65b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:51.599478 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=61 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:21:51.616384 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b267bebb-e197-4b2b-acc8-c8a3a02ec65b None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:21:51.617883 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b267bebb-e197-4b2b-acc8-c8a3a02ec65b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:21:51.627322 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 715/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 18:21:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:21:51.672164 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e294b4-f5ea-479a-b8a4-85f4008faddc None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:21:51.686032 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e294b4-f5ea-479a-b8a4-85f4008faddc None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:21:51.690194 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f22a09-8249-48e4-8178-114924f671c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:21:51.690908 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f22a09-8249-48e4-8178-114924f671c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:51.691247 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f22a09-8249-48e4-8178-114924f671c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:51.691567 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f22a09-8249-48e4-8178-114924f671c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:51.703654 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f22a09-8249-48e4-8178-114924f671c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:21:51.705332 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f22a09-8249-48e4-8178-114924f671c0 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:21:51.716958 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 737/1452] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:21:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:21:54.126049 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63022b7-cd6c-4d7d-a742-18d910c7f027 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:54.126620 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63022b7-cd6c-4d7d-a742-18d910c7f027 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:54.126620 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63022b7-cd6c-4d7d-a742-18d910c7f027 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:54.128177 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 716/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 18:21:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:21:54.136786 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c66647-5604-4ff7-bec6-72fc16c103ec None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:54.137428 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c66647-5604-4ff7-bec6-72fc16c103ec None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:54.137833 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c66647-5604-4ff7-bec6-72fc16c103ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:54.173421 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-95c66647-5604-4ff7-bec6-72fc16c103ec None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:21:54.175560 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-95c66647-5604-4ff7-bec6-72fc16c103ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:21:54.190193 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 738/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 18:21:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:21:54.234023 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63022b7-cd6c-4d7d-a742-18d910c7f027 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:21:54.253692 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63022b7-cd6c-4d7d-a742-18d910c7f027 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:21:54.264075 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f038988-f1d4-4a3e-bdff-d60e805398b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:21:54.265179 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f038988-f1d4-4a3e-bdff-d60e805398b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:54.266260 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f038988-f1d4-4a3e-bdff-d60e805398b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:54.266260 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f038988-f1d4-4a3e-bdff-d60e805398b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:54.292854 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f038988-f1d4-4a3e-bdff-d60e805398b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:21:54.295647 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f038988-f1d4-4a3e-bdff-d60e805398b3 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:21:54.310764 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 717/1455] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:21:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:21:56.910977 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f610a-4dad-4d34-8a1c-d55235e55bdb None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:56.912549 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f610a-4dad-4d34-8a1c-d55235e55bdb None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:56.912549 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f610a-4dad-4d34-8a1c-d55235e55bdb None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:56.913977 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 739/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 18:21:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:21:56.920347 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d3b6b0-8eef-4fbb-b18b-9eb73911a576 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:56.921009 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d3b6b0-8eef-4fbb-b18b-9eb73911a576 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:56.921009 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d3b6b0-8eef-4fbb-b18b-9eb73911a576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:56.941580 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-59d3b6b0-8eef-4fbb-b18b-9eb73911a576 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:21:56.943371 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-59d3b6b0-8eef-4fbb-b18b-9eb73911a576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:21:56.952689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 718/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 18:21:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:21:56.997376 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503f610a-4dad-4d34-8a1c-d55235e55bdb None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:21:57.010545 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503f610a-4dad-4d34-8a1c-d55235e55bdb None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:21:57.014687 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa4e24d-7347-41d6-bfbe-1856f91af71f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:21:57.015338 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4e24d-7347-41d6-bfbe-1856f91af71f service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:57.015619 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4e24d-7347-41d6-bfbe-1856f91af71f service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:57.015894 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4e24d-7347-41d6-bfbe-1856f91af71f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:57.032386 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa4e24d-7347-41d6-bfbe-1856f91af71f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:21:57.035170 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa4e24d-7347-41d6-bfbe-1856f91af71f service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:21:57.054454 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 740/1458] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:21:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:21:59.415874 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241a6b5-b8fe-4de6-b2e2-600b149371be None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:59.416532 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241a6b5-b8fe-4de6-b2e2-600b149371be None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:59.416667 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241a6b5-b8fe-4de6-b2e2-600b149371be None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:59.418469 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 719/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 18:21:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:21:59.428541 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c24fd-9069-4957-a363-0e67aa320f4c None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:59.429176 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c24fd-9069-4957-a363-0e67aa320f4c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:59.429457 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c24fd-9069-4957-a363-0e67aa320f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:59.437830 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=70 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:21:59.461166 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-867c24fd-9069-4957-a363-0e67aa320f4c None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:21:59.462717 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-867c24fd-9069-4957-a363-0e67aa320f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:21:59.472541 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 741/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 18:21:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:21:59.520917 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6241a6b5-b8fe-4de6-b2e2-600b149371be None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:21:59.539267 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6241a6b5-b8fe-4de6-b2e2-600b149371be None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:21:59.546490 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa410e-1345-4b65-8799-6f5ac490864d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:21:59.546790 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa410e-1345-4b65-8799-6f5ac490864d service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:59.547244 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa410e-1345-4b65-8799-6f5ac490864d service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:59.547741 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa410e-1345-4b65-8799-6f5ac490864d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:59.566262 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa410e-1345-4b65-8799-6f5ac490864d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:21:59.570155 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa410e-1345-4b65-8799-6f5ac490864d service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:21:59.590507 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 720/1461] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:21:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:22:02.137402 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329b997-b293-4588-8f5f-12620251112a None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:22:02.138038 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329b997-b293-4588-8f5f-12620251112a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:22:02.138038 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329b997-b293-4588-8f5f-12620251112a None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:22:02.139395 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 742/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 18:22:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:22:02.148615 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8c43a-c8e6-43a3-9ba0-3f03375068e0 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:22:02.149442 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8c43a-c8e6-43a3-9ba0-3f03375068e0 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:22:02.149673 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8c43a-c8e6-43a3-9ba0-3f03375068e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:22:02.170053 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a8c8c43a-c8e6-43a3-9ba0-3f03375068e0 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:22:02.171470 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a8c8c43a-c8e6-43a3-9ba0-3f03375068e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:22:02.181141 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 721/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 18:22:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:22:02.226731 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4329b997-b293-4588-8f5f-12620251112a None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:22:02.239790 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4329b997-b293-4588-8f5f-12620251112a None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:22:02.243667 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbadce2-8f21-4a01-9eb3-ee8f9fbb3579 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:22:02.244343 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbadce2-8f21-4a01-9eb3-ee8f9fbb3579 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:22:02.244627 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbadce2-8f21-4a01-9eb3-ee8f9fbb3579 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:22:02.244907 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbadce2-8f21-4a01-9eb3-ee8f9fbb3579 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:22:02.257355 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbadce2-8f21-4a01-9eb3-ee8f9fbb3579 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:22:02.260277 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbadce2-8f21-4a01-9eb3-ee8f9fbb3579 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:22:02.273419 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 743/1464] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:22:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:22:04.693118 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35513a88-3004-4b3f-8549-7919fd47a89b None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:22:04.693768 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35513a88-3004-4b3f-8549-7919fd47a89b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:22:04.694037 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35513a88-3004-4b3f-8549-7919fd47a89b None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:22:04.696804 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 722/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 18:22:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:22:04.703229 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b05f99-8831-4921-a643-8212a7664dff None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:22:04.703970 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b05f99-8831-4921-a643-8212a7664dff None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:22:04.704126 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b05f99-8831-4921-a643-8212a7664dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:22:04.730563 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-93b05f99-8831-4921-a643-8212a7664dff None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:22:04.732578 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-93b05f99-8831-4921-a643-8212a7664dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:22:04.747494 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 744/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 18:22:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:22:04.786656 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35513a88-3004-4b3f-8549-7919fd47a89b None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:22:04.800221 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35513a88-3004-4b3f-8549-7919fd47a89b None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:22:04.804548 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a409bdca-a2e5-4ad2-94cc-bcff279a25f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:22:04.805293 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409bdca-a2e5-4ad2-94cc-bcff279a25f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:22:04.805828 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409bdca-a2e5-4ad2-94cc-bcff279a25f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:22:04.806180 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409bdca-a2e5-4ad2-94cc-bcff279a25f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:22:04.819616 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a409bdca-a2e5-4ad2-94cc-bcff279a25f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:22:04.822456 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a409bdca-a2e5-4ad2-94cc-bcff279a25f1 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:22:04.841681 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 723/1467] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:22:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:22:07.172657 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8dcc33-8663-4177-af05-5c1411fcf9ba None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:22:07.173286 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8dcc33-8663-4177-af05-5c1411fcf9ba None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:22:07.173494 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8dcc33-8663-4177-af05-5c1411fcf9ba None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:22:07.176194 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 745/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 18:22:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:22:07.183644 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f2a92-7ebc-4c59-9e2a-619f76af99f7 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:22:07.184543 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f2a92-7ebc-4c59-9e2a-619f76af99f7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:22:07.184543 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f2a92-7ebc-4c59-9e2a-619f76af99f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:22:07.206534 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-aa1f2a92-7ebc-4c59-9e2a-619f76af99f7 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:22:07.208172 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-aa1f2a92-7ebc-4c59-9e2a-619f76af99f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:22:07.217319 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 724/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 18:22:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:22:07.257188 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8dcc33-8663-4177-af05-5c1411fcf9ba None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:22:07.273522 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8dcc33-8663-4177-af05-5c1411fcf9ba None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:22:07.277806 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f4d53b-b80f-4240-89c8-3d29a5c134bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:22:07.278615 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4d53b-b80f-4240-89c8-3d29a5c134bb service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:22:07.278923 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4d53b-b80f-4240-89c8-3d29a5c134bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:22:07.279198 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4d53b-b80f-4240-89c8-3d29a5c134bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:22:07.290279 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f4d53b-b80f-4240-89c8-3d29a5c134bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:22:07.292140 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f4d53b-b80f-4240-89c8-3d29a5c134bb service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:22:07.304099 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 746/1470] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:22:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:22:17.200217 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=29 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:22:17.299007 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=17 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:22:33.401679 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d079c0c-47e2-4e56-9576-ab5a80befdf4 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:22:33.402493 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d079c0c-47e2-4e56-9576-ab5a80befdf4 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:22:33.402493 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d079c0c-47e2-4e56-9576-ab5a80befdf4 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:22:33.403863 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 725/1471] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:22:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:22:33.410739 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ab1fc-e5ee-4271-8ff9-b96faf993f0a None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:22:33.411608 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ab1fc-e5ee-4271-8ff9-b96faf993f0a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:22:33.411608 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ab1fc-e5ee-4271-8ff9-b96faf993f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:22:33.435264 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-955ab1fc-e5ee-4271-8ff9-b96faf993f0a None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:22:33.437925 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-955ab1fc-e5ee-4271-8ff9-b96faf993f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:22:33.455089 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 747/1472] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:22:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:22:33.518859 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec531c51-605e-4360-8e2c-eeee9b6cf132 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:22:33.519638 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec531c51-605e-4360-8e2c-eeee9b6cf132 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:22:33.520140 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec531c51-605e-4360-8e2c-eeee9b6cf132 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:22:33.522964 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 726/1473] 10.222.0.87 () {54 vars in 962 bytes} [Tue Jun 21 18:22:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:22:33.529123 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bdedd8-4ef2-4fea-8761-bf3194767ddf None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:22:33.529479 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bdedd8-4ef2-4fea-8761-bf3194767ddf None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:22:33.529793 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bdedd8-4ef2-4fea-8761-bf3194767ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:22:33.562548 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-b9bdedd8-4ef2-4fea-8761-bf3194767ddf None None] MFA Rules not processed for user `e7da4bf6f3be47b4aa25c6240cb4be03`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:22:33.563915 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b9bdedd8-4ef2-4fea-8761-bf3194767ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:22:33.571370 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 748/1474] 10.222.0.87 () {58 vars in 1059 bytes} [Tue Jun 21 18:22:33 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:22:33.580120 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec531c51-605e-4360-8e2c-eeee9b6cf132 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:22:33.603901 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec531c51-605e-4360-8e2c-eeee9b6cf132 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:22:33.610263 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b3672b-6249-48e8-af7a-32ff34e8492c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7da4bf6f3be47b4aa25c6240cb4be03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:22:33.611382 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b3672b-6249-48e8-af7a-32ff34e8492c service glance] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:22:33.611837 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b3672b-6249-48e8-af7a-32ff34e8492c service glance] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:22:33.612286 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b3672b-6249-48e8-af7a-32ff34e8492c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:22:33.629183 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b3672b-6249-48e8-af7a-32ff34e8492c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:22:33.631967 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b3672b-6249-48e8-af7a-32ff34e8492c service glance] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:22:33.651368 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 727/1475] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:22:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:22:36.365611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6239315-e827-49a2-a33b-be70819d0c8c None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:22:36.366409 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6239315-e827-49a2-a33b-be70819d0c8c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:22:36.366605 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6239315-e827-49a2-a33b-be70819d0c8c None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:22:36.369289 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 749/1476] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:22:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:22:36.380573 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c2eb8d-001e-483e-851d-2fbd6182ecd6 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:22:36.381374 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c2eb8d-001e-483e-851d-2fbd6182ecd6 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:22:36.381553 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c2eb8d-001e-483e-851d-2fbd6182ecd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:22:36.407259 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-c8c2eb8d-001e-483e-851d-2fbd6182ecd6 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:22:36.409993 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c8c2eb8d-001e-483e-851d-2fbd6182ecd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:22:36.424263 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 728/1477] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:22:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:22:36.467402 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6239315-e827-49a2-a33b-be70819d0c8c None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:22:36.491066 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6239315-e827-49a2-a33b-be70819d0c8c None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:22:36.500330 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9430b3-7738-4928-941d-0875326163f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:22:36.501419 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9430b3-7738-4928-941d-0875326163f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:22:36.502030 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9430b3-7738-4928-941d-0875326163f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:22:36.502570 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9430b3-7738-4928-941d-0875326163f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:22:36.504252 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9430b3-7738-4928-941d-0875326163f1 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:22:36.507439 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: WARNING py.warnings [None req-9e9430b3-7738-4928-941d-0875326163f1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 18:22:36.507439 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: warnings.warn(msg) Jun 21 18:22:36.507439 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: Jun 21 18:22:36.509473 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9430b3-7738-4928-941d-0875326163f1 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:22:36.540104 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 750/1478] 10.222.0.87 () {56 vars in 1108 bytes} [Tue Jun 21 18:22:36 2022] GET /identity/v3/projects => generated 2553 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:22:39.108934 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da68c5e8-a7af-44ab-80e2-5a70e7758bc5 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:22:39.109913 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da68c5e8-a7af-44ab-80e2-5a70e7758bc5 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:22:39.109913 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da68c5e8-a7af-44ab-80e2-5a70e7758bc5 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:22:39.111449 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 729/1479] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:22:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:22:39.118858 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d39fb8a-01d3-4715-acc0-a3da074235bb None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:22:39.119922 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d39fb8a-01d3-4715-acc0-a3da074235bb None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:22:39.119922 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d39fb8a-01d3-4715-acc0-a3da074235bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:22:39.148222 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-2d39fb8a-01d3-4715-acc0-a3da074235bb None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:22:39.150449 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-2d39fb8a-01d3-4715-acc0-a3da074235bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:22:39.167927 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 751/1480] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:22:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:22:39.238498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da68c5e8-a7af-44ab-80e2-5a70e7758bc5 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:22:39.261504 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da68c5e8-a7af-44ab-80e2-5a70e7758bc5 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:22:39.269466 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f54eb-4273-4ecb-9632-2d38950b92a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:22:39.270262 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f54eb-4273-4ecb-9632-2d38950b92a7 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:22:39.270595 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f54eb-4273-4ecb-9632-2d38950b92a7 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:22:39.270925 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f54eb-4273-4ecb-9632-2d38950b92a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:22:39.281619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90f54eb-4273-4ecb-9632-2d38950b92a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:22:39.283532 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90f54eb-4273-4ecb-9632-2d38950b92a7 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:22:39.295516 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 730/1481] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:22:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:22:42.541096 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cbcd4e-1dfb-4f98-8806-3df6e82cb49a None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:22:42.542131 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cbcd4e-1dfb-4f98-8806-3df6e82cb49a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:22:42.542131 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cbcd4e-1dfb-4f98-8806-3df6e82cb49a None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:22:42.543690 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 752/1482] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:22:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:22:42.552834 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7baed11-0181-4a28-ae1d-b1589206a070 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:22:42.553774 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7baed11-0181-4a28-ae1d-b1589206a070 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:22:42.553774 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7baed11-0181-4a28-ae1d-b1589206a070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:22:42.576950 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f7baed11-0181-4a28-ae1d-b1589206a070 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:22:42.578816 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f7baed11-0181-4a28-ae1d-b1589206a070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:22:42.589386 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 731/1483] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:22:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:22:42.635860 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cbcd4e-1dfb-4f98-8806-3df6e82cb49a None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:22:42.652194 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cbcd4e-1dfb-4f98-8806-3df6e82cb49a None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:22:42.657762 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f1aaf-8e8c-48c9-ad11-0db3e8794f23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:22:42.658580 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f1aaf-8e8c-48c9-ad11-0db3e8794f23 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:22:42.659241 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f1aaf-8e8c-48c9-ad11-0db3e8794f23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:22:42.659884 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f1aaf-8e8c-48c9-ad11-0db3e8794f23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:22:42.677327 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616f1aaf-8e8c-48c9-ad11-0db3e8794f23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:22:42.680202 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616f1aaf-8e8c-48c9-ad11-0db3e8794f23 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:22:42.697931 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 753/1484] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:22:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:22:45.387761 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0c36e-daab-488f-8601-23e449559600 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:22:45.388678 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0c36e-daab-488f-8601-23e449559600 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:22:45.388678 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0c36e-daab-488f-8601-23e449559600 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:22:45.389850 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 732/1485] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:22:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:22:45.397866 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b3bf1-5409-454e-aea6-c7976e65ff2c None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:22:45.398806 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b3bf1-5409-454e-aea6-c7976e65ff2c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:22:45.398806 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b3bf1-5409-454e-aea6-c7976e65ff2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:22:45.422118 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-744b3bf1-5409-454e-aea6-c7976e65ff2c None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:22:45.423511 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-744b3bf1-5409-454e-aea6-c7976e65ff2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:22:45.432977 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 754/1486] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:22:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:22:45.476579 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee0c36e-daab-488f-8601-23e449559600 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:22:45.491319 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee0c36e-daab-488f-8601-23e449559600 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:22:45.495950 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64a52ec-3fd6-4b22-9455-a7646ec42b9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:22:45.496683 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a52ec-3fd6-4b22-9455-a7646ec42b9d service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:22:45.497039 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a52ec-3fd6-4b22-9455-a7646ec42b9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:22:45.497426 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a52ec-3fd6-4b22-9455-a7646ec42b9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:22:45.509074 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64a52ec-3fd6-4b22-9455-a7646ec42b9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:22:45.510839 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64a52ec-3fd6-4b22-9455-a7646ec42b9d service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:22:45.524905 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 733/1487] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:22:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:22:48.783481 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5c39f-f985-4d6c-a5ea-26f332dbf50b None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:22:48.784897 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5c39f-f985-4d6c-a5ea-26f332dbf50b None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:22:48.784897 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5c39f-f985-4d6c-a5ea-26f332dbf50b None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:22:48.786437 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 755/1488] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:22:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:22:48.793631 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d911c5-8b50-4709-ba84-5451dad08345 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:22:48.794343 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d911c5-8b50-4709-ba84-5451dad08345 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:22:48.794343 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d911c5-8b50-4709-ba84-5451dad08345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:22:48.822481 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-e1d911c5-8b50-4709-ba84-5451dad08345 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:22:48.824421 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e1d911c5-8b50-4709-ba84-5451dad08345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:22:48.835833 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 734/1489] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:22:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:22:48.890536 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5c39f-f985-4d6c-a5ea-26f332dbf50b None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:22:48.905315 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5c39f-f985-4d6c-a5ea-26f332dbf50b None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:22:48.910529 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d373e1d8-0118-48db-a658-dcf8aa43d799 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:22:48.911690 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373e1d8-0118-48db-a658-dcf8aa43d799 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:22:48.911916 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373e1d8-0118-48db-a658-dcf8aa43d799 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:22:48.912317 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373e1d8-0118-48db-a658-dcf8aa43d799 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:22:48.925752 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d373e1d8-0118-48db-a658-dcf8aa43d799 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:22:48.928514 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d373e1d8-0118-48db-a658-dcf8aa43d799 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:22:48.941210 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 756/1490] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:22:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:22:52.165360 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d1d7e3-4919-4c98-8a0d-8626261f92b0 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:22:52.166351 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d1d7e3-4919-4c98-8a0d-8626261f92b0 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:22:52.166351 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d1d7e3-4919-4c98-8a0d-8626261f92b0 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:22:52.168995 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 735/1491] 10.222.0.87 () {54 vars in 926 bytes} [Tue Jun 21 18:22:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:22:52.177828 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801e9ba-668e-4fe0-8546-64423e18f203 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:22:52.178743 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801e9ba-668e-4fe0-8546-64423e18f203 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:22:52.178743 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801e9ba-668e-4fe0-8546-64423e18f203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:22:52.212201 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-a801e9ba-668e-4fe0-8546-64423e18f203 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:22:52.214354 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a801e9ba-668e-4fe0-8546-64423e18f203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:22:52.229887 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 757/1492] 10.222.0.87 () {58 vars in 1023 bytes} [Tue Jun 21 18:22:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:22:52.273769 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d1d7e3-4919-4c98-8a0d-8626261f92b0 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:22:52.290280 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d1d7e3-4919-4c98-8a0d-8626261f92b0 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:22:52.295096 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fad97b-b366-407d-9521-396d58ee61ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:22:52.296115 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fad97b-b366-407d-9521-396d58ee61ee service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:22:52.296586 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fad97b-b366-407d-9521-396d58ee61ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:22:52.297030 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fad97b-b366-407d-9521-396d58ee61ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:22:52.316061 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fad97b-b366-407d-9521-396d58ee61ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:22:52.318657 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fad97b-b366-407d-9521-396d58ee61ee service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:22:52.334645 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 736/1493] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:22:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:23:02.201943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=33 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:23:02.327198 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=33 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:24:53.435132 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b4331-68af-451b-b802-521416ac85a7 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:24:53.435956 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b4331-68af-451b-b802-521416ac85a7 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:24:53.435956 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b4331-68af-451b-b802-521416ac85a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:24:53.458367 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-ee5b4331-68af-451b-b802-521416ac85a7 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:24:53.459726 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ee5b4331-68af-451b-b802-521416ac85a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:24:53.470500 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 758/1494] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:24:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:24:53.482185 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27825d6-1604-4a27-a156-c0c5849c9ee4 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:24:53.483089 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27825d6-1604-4a27-a156-c0c5849c9ee4 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:24:53.483089 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27825d6-1604-4a27-a156-c0c5849c9ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:24:53.513678 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a27825d6-1604-4a27-a156-c0c5849c9ee4 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:24:53.514947 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a27825d6-1604-4a27-a156-c0c5849c9ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:24:53.525763 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 737/1495] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:24:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:24:53.531837 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b4331-68af-451b-b802-521416ac85a7 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:24:53.544406 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b4331-68af-451b-b802-521416ac85a7 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:24:53.551104 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148976af-a8a6-498c-bbbd-57661a99c4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:24:53.552116 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148976af-a8a6-498c-bbbd-57661a99c4af admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:24:53.552426 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148976af-a8a6-498c-bbbd-57661a99c4af admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:24:53.552790 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148976af-a8a6-498c-bbbd-57661a99c4af admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:24:53.553733 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148976af-a8a6-498c-bbbd-57661a99c4af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:24:53.554075 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148976af-a8a6-498c-bbbd-57661a99c4af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:24:53.555420 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: WARNING py.warnings [None req-148976af-a8a6-498c-bbbd-57661a99c4af admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 21 18:24:53.555420 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: warnings.warn(msg) Jun 21 18:24:53.555420 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: Jun 21 18:24:53.556389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148976af-a8a6-498c-bbbd-57661a99c4af admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:24:53.573375 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 759/1496] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:24:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:24:53.584245 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27825d6-1604-4a27-a156-c0c5849c9ee4 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:24:53.599943 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27825d6-1604-4a27-a156-c0c5849c9ee4 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:24:53.609512 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cccf50a-241c-44a5-ae81-7d0da1ac3ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:24:53.611011 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cccf50a-241c-44a5-ae81-7d0da1ac3ec2 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:24:53.611499 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cccf50a-241c-44a5-ae81-7d0da1ac3ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:24:53.611981 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cccf50a-241c-44a5-ae81-7d0da1ac3ec2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:24:53.614050 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cccf50a-241c-44a5-ae81-7d0da1ac3ec2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:24:53.617352 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: WARNING py.warnings [None req-7cccf50a-241c-44a5-ae81-7d0da1ac3ec2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 18:24:53.617352 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: warnings.warn(msg) Jun 21 18:24:53.617352 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: Jun 21 18:24:53.619758 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cccf50a-241c-44a5-ae81-7d0da1ac3ec2 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:24:53.679960 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 738/1497] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:24:53 2022] POST /identity/v3/projects => generated 379 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:24:53.685104 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148976af-a8a6-498c-bbbd-57661a99c4af admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:24:53.698174 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148976af-a8a6-498c-bbbd-57661a99c4af admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:24:53.811786 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedfd801-1f59-40b3-869e-c29444d111ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:24:53.812502 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedfd801-1f59-40b3-869e-c29444d111ec admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:24:53.812810 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedfd801-1f59-40b3-869e-c29444d111ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:24:53.813128 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedfd801-1f59-40b3-869e-c29444d111ec admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:24:53.814003 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedfd801-1f59-40b3-869e-c29444d111ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:24:53.815133 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: WARNING py.warnings [None req-eedfd801-1f59-40b3-869e-c29444d111ec admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 18:24:53.815133 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: warnings.warn(msg) Jun 21 18:24:53.815133 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: Jun 21 18:24:53.816683 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedfd801-1f59-40b3-869e-c29444d111ec admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:24:53.848538 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 760/1498] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:24:53 2022] POST /identity/v3/users => generated 290 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:24:53.854380 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cccf50a-241c-44a5-ae81-7d0da1ac3ec2 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:24:53.874102 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cccf50a-241c-44a5-ae81-7d0da1ac3ec2 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:24:53.882616 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5e3e4b-feb8-426f-a7fa-86046899a6a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:24:53.883794 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e3e4b-feb8-426f-a7fa-86046899a6a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:24:53.884285 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e3e4b-feb8-426f-a7fa-86046899a6a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:24:53.884776 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e3e4b-feb8-426f-a7fa-86046899a6a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:24:53.886342 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5e3e4b-feb8-426f-a7fa-86046899a6a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:24:53.888337 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: WARNING py.warnings [None req-4c5e3e4b-feb8-426f-a7fa-86046899a6a7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 21 18:24:53.888337 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: warnings.warn(msg) Jun 21 18:24:53.888337 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: Jun 21 18:24:53.890140 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5e3e4b-feb8-426f-a7fa-86046899a6a7 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:24:53.916966 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 739/1499] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:24:53.924008 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedfd801-1f59-40b3-869e-c29444d111ec admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:24:53.938009 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedfd801-1f59-40b3-869e-c29444d111ec admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:24:53.944829 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7c3f51-9d4e-4e53-942a-f9f9a7238522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:24:53.945679 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c3f51-9d4e-4e53-942a-f9f9a7238522 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:24:53.946026 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c3f51-9d4e-4e53-942a-f9f9a7238522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:24:53.946360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c3f51-9d4e-4e53-942a-f9f9a7238522 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:24:53.947177 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7c3f51-9d4e-4e53-942a-f9f9a7238522 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:24:53.948420 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: WARNING py.warnings [None req-4d7c3f51-9d4e-4e53-942a-f9f9a7238522 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 21 18:24:53.948420 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: warnings.warn(msg) Jun 21 18:24:53.948420 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: Jun 21 18:24:53.949792 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7c3f51-9d4e-4e53-942a-f9f9a7238522 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:24:53.971478 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 761/1500] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:24:53.978298 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5e3e4b-feb8-426f-a7fa-86046899a6a7 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:24:53.998839 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5e3e4b-feb8-426f-a7fa-86046899a6a7 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:24:54.007810 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbbfc87-8dca-4807-be56-93974b90abd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:24:54.009384 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbbfc87-8dca-4807-be56-93974b90abd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:24:54.009940 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbbfc87-8dca-4807-be56-93974b90abd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:24:54.010466 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbbfc87-8dca-4807-be56-93974b90abd7 admin admin] PATH_INFO: `/v3/projects/d06c485de09b44e29a7d654d2dbe470b/users/fa5dd7e977114997be995fa2deede80e/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:24:54.035895 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbbfc87-8dca-4807-be56-93974b90abd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d06c485de09b44e29a7d654d2dbe470b, user_id=fa5dd7e977114997be995fa2deede80e, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:24:54.037238 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: WARNING py.warnings [None req-8dbbfc87-8dca-4807-be56-93974b90abd7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 18:24:54.037238 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: warnings.warn(msg) Jun 21 18:24:54.037238 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: Jun 21 18:24:54.040782 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbbfc87-8dca-4807-be56-93974b90abd7 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:24:54.093938 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 740/1501] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:24:53 2022] PUT /identity/v3/projects/d06c485de09b44e29a7d654d2dbe470b/users/fa5dd7e977114997be995fa2deede80e/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:24:54.119033 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3077a86-f05e-4c75-80d8-4a41daea7eb6 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:24:54.119557 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3077a86-f05e-4c75-80d8-4a41daea7eb6 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:24:54.119982 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3077a86-f05e-4c75-80d8-4a41daea7eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:24:54.171236 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-f3077a86-f05e-4c75-80d8-4a41daea7eb6 None None] MFA Rules not processed for user `fa5dd7e977114997be995fa2deede80e`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:24:54.173254 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f3077a86-f05e-4c75-80d8-4a41daea7eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:24:54.265748 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 762/1502] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:24:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:24:54.289595 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26299046-c626-4300-8cef-61a2d24ef2d8 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:24:54.290097 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26299046-c626-4300-8cef-61a2d24ef2d8 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:24:54.290471 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26299046-c626-4300-8cef-61a2d24ef2d8 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:24:54.292838 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 741/1503] 10.222.0.87 () {54 vars in 963 bytes} [Tue Jun 21 18:24:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:24:54.299254 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44c8c7-eb36-4e7d-b2f3-8ff1069fab72 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:24:54.299576 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44c8c7-eb36-4e7d-b2f3-8ff1069fab72 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:24:54.299895 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44c8c7-eb36-4e7d-b2f3-8ff1069fab72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:24:54.323230 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-8d44c8c7-eb36-4e7d-b2f3-8ff1069fab72 None None] MFA Rules not processed for user `b683e6f78abd4986b415eb9ddf34cb8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:24:54.324649 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8d44c8c7-eb36-4e7d-b2f3-8ff1069fab72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:24:54.395128 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:24:54.406361 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 763/1504] 10.222.0.87 () {58 vars in 1060 bytes} [Tue Jun 21 18:24:54 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:24:54.420177 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26299046-c626-4300-8cef-61a2d24ef2d8 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:24:54.439839 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26299046-c626-4300-8cef-61a2d24ef2d8 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:24:54.447131 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f52f0e-f580-466e-8850-c17f59614e60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:24:54.448280 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f52f0e-f580-466e-8850-c17f59614e60 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:24:54.448789 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f52f0e-f580-466e-8850-c17f59614e60 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:24:54.449275 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f52f0e-f580-466e-8850-c17f59614e60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:24:54.466031 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f52f0e-f580-466e-8850-c17f59614e60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:24:54.468889 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f52f0e-f580-466e-8850-c17f59614e60 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:24:54.487841 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 742/1505] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:24:54 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:24:54.553424 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de1f2ce-0d1f-4068-9017-79ed37036abb None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:24:54.554721 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de1f2ce-0d1f-4068-9017-79ed37036abb None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:24:54.555180 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de1f2ce-0d1f-4068-9017-79ed37036abb None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:24:54.557557 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 764/1506] 10.222.0.87 () {52 vars in 829 bytes} [Tue Jun 21 18:24:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:24:54.563752 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f52f0e-f580-466e-8850-c17f59614e60 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:24:54.657153 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f52f0e-f580-466e-8850-c17f59614e60 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:24:54.664841 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f1c65d-acd0-4c29-ad47-7e9733450c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:24:54.665789 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1c65d-acd0-4c29-ad47-7e9733450c7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:24:54.666081 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1c65d-acd0-4c29-ad47-7e9733450c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:24:54.666364 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1c65d-acd0-4c29-ad47-7e9733450c7d admin admin] PATH_INFO: `/v3/users/fa5dd7e977114997be995fa2deede80e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:24:54.667712 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f1c65d-acd0-4c29-ad47-7e9733450c7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa5dd7e977114997be995fa2deede80e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:24:54.668780 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: WARNING py.warnings [None req-d2f1c65d-acd0-4c29-ad47-7e9733450c7d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 18:24:54.668780 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: warnings.warn(msg) Jun 21 18:24:54.668780 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: Jun 21 18:24:54.670046 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f1c65d-acd0-4c29-ad47-7e9733450c7d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:24:54.798138 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d2f1c65d-acd0-4c29-ad47-7e9733450c7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa5dd7e977114997be995fa2deede80e'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:24:54.828867 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d2f1c65d-acd0-4c29-ad47-7e9733450c7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa5dd7e977114997be995fa2deede80e'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:24:54.830259 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d2f1c65d-acd0-4c29-ad47-7e9733450c7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa5dd7e977114997be995fa2deede80e'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:24:54.831395 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d2f1c65d-acd0-4c29-ad47-7e9733450c7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa5dd7e977114997be995fa2deede80e'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:24:54.862483 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d2f1c65d-acd0-4c29-ad47-7e9733450c7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa5dd7e977114997be995fa2deede80e'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:24:54.869060 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:24:54.885271 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 743/1507] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:24:54 2022] DELETE /identity/v3/users/fa5dd7e977114997be995fa2deede80e => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:24:54.890520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3e804-13fe-4232-ad63-03f9e8bac16b None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:24:54.890822 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3e804-13fe-4232-ad63-03f9e8bac16b None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:24:54.891071 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3e804-13fe-4232-ad63-03f9e8bac16b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:24:54.913711 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-5fc3e804-13fe-4232-ad63-03f9e8bac16b None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:24:54.914844 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5fc3e804-13fe-4232-ad63-03f9e8bac16b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:24:55.016195 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 765/1508] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:24:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:24:55.030252 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f1c65d-acd0-4c29-ad47-7e9733450c7d admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:24:55.031941 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d2f1c65d-acd0-4c29-ad47-7e9733450c7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:24:55.125863 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f1c65d-acd0-4c29-ad47-7e9733450c7d admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:24:55.133487 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ba84d-4b72-445d-8bd3-4d404bf04537 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:24:55.134682 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ba84d-4b72-445d-8bd3-4d404bf04537 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:24:55.135180 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ba84d-4b72-445d-8bd3-4d404bf04537 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:24:55.135672 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ba84d-4b72-445d-8bd3-4d404bf04537 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:24:55.148555 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849ba84d-4b72-445d-8bd3-4d404bf04537 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:24:55.150576 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849ba84d-4b72-445d-8bd3-4d404bf04537 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:24:55.165781 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 744/1509] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:24:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:24:55.685442 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3e804-13fe-4232-ad63-03f9e8bac16b None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:24:55.689910 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5fc3e804-13fe-4232-ad63-03f9e8bac16b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:24:55.714586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3e804-13fe-4232-ad63-03f9e8bac16b None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:24:55.724313 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba089697-f910-42d7-9a8e-46a366cbf61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:24:55.725301 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba089697-f910-42d7-9a8e-46a366cbf61e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:24:55.725622 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba089697-f910-42d7-9a8e-46a366cbf61e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:24:55.726339 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba089697-f910-42d7-9a8e-46a366cbf61e admin admin] PATH_INFO: `/v3/projects/d06c485de09b44e29a7d654d2dbe470b` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:24:55.727895 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba089697-f910-42d7-9a8e-46a366cbf61e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d06c485de09b44e29a7d654d2dbe470b)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:24:55.729026 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: WARNING py.warnings [None req-ba089697-f910-42d7-9a8e-46a366cbf61e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 18:24:55.729026 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: warnings.warn(msg) Jun 21 18:24:55.729026 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: Jun 21 18:24:55.730367 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba089697-f910-42d7-9a8e-46a366cbf61e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:24:55.824415 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-ba089697-f910-42d7-9a8e-46a366cbf61e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd06c485de09b44e29a7d654d2dbe470b', 'request_id': 'req-ba089697-f910-42d7-9a8e-46a366cbf61e'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:24:55.832191 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-ba089697-f910-42d7-9a8e-46a366cbf61e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd06c485de09b44e29a7d654d2dbe470b', 'request_id': 'req-ba089697-f910-42d7-9a8e-46a366cbf61e'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:24:55.845960 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-ba089697-f910-42d7-9a8e-46a366cbf61e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd06c485de09b44e29a7d654d2dbe470b', 'request_id': 'req-ba089697-f910-42d7-9a8e-46a366cbf61e'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:24:55.883148 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-ba089697-f910-42d7-9a8e-46a366cbf61e admin admin] The token cache is being invalidate because project d06c485de09b44e29a7d654d2dbe470b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:24:55.883956 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-ba089697-f910-42d7-9a8e-46a366cbf61e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:24:55.884838 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-ba089697-f910-42d7-9a8e-46a366cbf61e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:24:55.887876 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 766/1510] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:24:55 2022] DELETE /identity/v3/projects/d06c485de09b44e29a7d654d2dbe470b => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:25:05.159442 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:25:05.876708 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:26:45.127345 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81351f4b-1f7f-43e5-a1d6-ed3718e074b9 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:45.128258 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81351f4b-1f7f-43e5-a1d6-ed3718e074b9 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:45.128258 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81351f4b-1f7f-43e5-a1d6-ed3718e074b9 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:45.131119 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 745/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 18:26:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:26:45.138160 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4307008a-a564-4f42-aefa-f13a59a02b1a None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:45.139736 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4307008a-a564-4f42-aefa-f13a59a02b1a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:45.140581 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4307008a-a564-4f42-aefa-f13a59a02b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:45.163884 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-4307008a-a564-4f42-aefa-f13a59a02b1a None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:26:45.165841 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4307008a-a564-4f42-aefa-f13a59a02b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:26:45.291490 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 767/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 18:26:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:26:45.576222 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81351f4b-1f7f-43e5-a1d6-ed3718e074b9 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:45.579173 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-81351f4b-1f7f-43e5-a1d6-ed3718e074b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:26:45.692505 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81351f4b-1f7f-43e5-a1d6-ed3718e074b9 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:45.699348 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66060361-6009-4516-be3c-a3fd65c49092 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:45.700094 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66060361-6009-4516-be3c-a3fd65c49092 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:45.700387 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66060361-6009-4516-be3c-a3fd65c49092 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:45.700681 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66060361-6009-4516-be3c-a3fd65c49092 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:45.714280 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66060361-6009-4516-be3c-a3fd65c49092 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:45.717441 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66060361-6009-4516-be3c-a3fd65c49092 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:45.736353 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 746/1513] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:26:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:26:46.362059 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188b8f1-0608-425a-b4ad-7961894e8d69 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:46.363063 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188b8f1-0608-425a-b4ad-7961894e8d69 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:46.363063 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188b8f1-0608-425a-b4ad-7961894e8d69 None None] PATH_INFO: `/v3` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:46.365865 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 768/1514] 10.222.0.75 () {54 vars in 925 bytes} [Tue Jun 21 18:26:46 2022] GET /identity/v3 => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:26:46.374989 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2770db2-9b7e-45d5-a772-dedf848e7657 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:46.375887 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2770db2-9b7e-45d5-a772-dedf848e7657 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:46.375887 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2770db2-9b7e-45d5-a772-dedf848e7657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:46.398028 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b2770db2-9b7e-45d5-a772-dedf848e7657 None None] MFA Rules not processed for user `ef1d49b36cdc4665b82487be97e9dcd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:26:46.399879 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b2770db2-9b7e-45d5-a772-dedf848e7657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:26:46.519696 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 747/1515] 10.222.0.75 () {58 vars in 1015 bytes} [Tue Jun 21 18:26:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:26:46.811271 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad2e92-2eef-49d2-94f8-a0a9c7a29efd None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:46.811844 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad2e92-2eef-49d2-94f8-a0a9c7a29efd None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:46.812182 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad2e92-2eef-49d2-94f8-a0a9c7a29efd None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:46.814609 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 769/1516] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 18:26:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:26:46.821733 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26792a13-c79c-4e4c-bb01-1d08fcf0a68c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:46.822146 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26792a13-c79c-4e4c-bb01-1d08fcf0a68c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:46.822518 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26792a13-c79c-4e4c-bb01-1d08fcf0a68c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:46.845572 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-26792a13-c79c-4e4c-bb01-1d08fcf0a68c None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:26:46.846923 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-26792a13-c79c-4e4c-bb01-1d08fcf0a68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:26:46.859637 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 748/1517] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 18:26:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:26:47.057144 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad2e92-2eef-49d2-94f8-a0a9c7a29efd None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:47.074579 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad2e92-2eef-49d2-94f8-a0a9c7a29efd None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:47.080613 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f940dba3-d8c2-4408-a5cf-9b4f963421c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:47.081287 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940dba3-d8c2-4408-a5cf-9b4f963421c7 service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:47.081573 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940dba3-d8c2-4408-a5cf-9b4f963421c7 service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:47.082123 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940dba3-d8c2-4408-a5cf-9b4f963421c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:47.094183 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f940dba3-d8c2-4408-a5cf-9b4f963421c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:47.096012 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f940dba3-d8c2-4408-a5cf-9b4f963421c7 service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:47.108661 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 770/1518] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:26:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:26:56.197218 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ab198-dab8-44ca-8e08-77c204c5aec3 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:56.197816 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ab198-dab8-44ca-8e08-77c204c5aec3 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:56.197917 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ab198-dab8-44ca-8e08-77c204c5aec3 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:56.199467 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 749/1519] 10.222.0.87 () {54 vars in 960 bytes} [Tue Jun 21 18:26:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:26:56.205664 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e69db-41e3-47bc-accd-33ba493ff0a9 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:56.206519 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e69db-41e3-47bc-accd-33ba493ff0a9 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:56.206682 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e69db-41e3-47bc-accd-33ba493ff0a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:56.227204 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-8c1e69db-41e3-47bc-accd-33ba493ff0a9 None None] MFA Rules not processed for user `ef1d49b36cdc4665b82487be97e9dcd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:26:56.228536 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8c1e69db-41e3-47bc-accd-33ba493ff0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:26:56.238886 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 771/1520] 10.222.0.87 () {58 vars in 1057 bytes} [Tue Jun 21 18:26:56 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:26:56.250026 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6ab198-dab8-44ca-8e08-77c204c5aec3 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:56.274583 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6ab198-dab8-44ca-8e08-77c204c5aec3 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:56.283424 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fba778-f7c1-450d-9853-74e2265ed31f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef1d49b36cdc4665b82487be97e9dcd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:56.284324 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fba778-f7c1-450d-9853-74e2265ed31f service placement] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:56.284794 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fba778-f7c1-450d-9853-74e2265ed31f service placement] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:56.285337 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fba778-f7c1-450d-9853-74e2265ed31f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:56.301445 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fba778-f7c1-450d-9853-74e2265ed31f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:56.303762 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fba778-f7c1-450d-9853-74e2265ed31f service placement] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:56.317447 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 750/1521] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:26:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:26:58.600849 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40022b6-2fff-4f04-8877-a108ef57756f None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:58.601622 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40022b6-2fff-4f04-8877-a108ef57756f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:58.601932 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40022b6-2fff-4f04-8877-a108ef57756f None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:58.604406 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 772/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 18:26:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:26:58.614021 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40dcff-a730-4b5c-99bd-19c8452599e4 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:58.614791 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40dcff-a730-4b5c-99bd-19c8452599e4 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:58.615024 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40dcff-a730-4b5c-99bd-19c8452599e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:58.646978 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-0d40dcff-a730-4b5c-99bd-19c8452599e4 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:26:58.649147 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0d40dcff-a730-4b5c-99bd-19c8452599e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:26:58.663661 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 751/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 18:26:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:26:58.862317 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40022b6-2fff-4f04-8877-a108ef57756f None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:58.885778 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40022b6-2fff-4f04-8877-a108ef57756f None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:58.895314 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11520bc-ca17-4237-9647-0866d678b214 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:58.896267 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11520bc-ca17-4237-9647-0866d678b214 service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:58.896588 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11520bc-ca17-4237-9647-0866d678b214 service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:58.896877 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11520bc-ca17-4237-9647-0866d678b214 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:58.911030 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11520bc-ca17-4237-9647-0866d678b214 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:58.914305 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11520bc-ca17-4237-9647-0866d678b214 service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:58.932080 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 773/1524] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:26:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:26:59.927449 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e170c-1655-4843-8cf0-95ea52fce9cf None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:59.928344 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e170c-1655-4843-8cf0-95ea52fce9cf None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:59.928553 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e170c-1655-4843-8cf0-95ea52fce9cf None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:59.930744 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 752/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 18:26:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:26:59.937544 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc88ee9-c7e3-49bc-b99e-1b1855fae733 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:59.938217 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc88ee9-c7e3-49bc-b99e-1b1855fae733 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:59.938217 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc88ee9-c7e3-49bc-b99e-1b1855fae733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:59.959101 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-acc88ee9-c7e3-49bc-b99e-1b1855fae733 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:26:59.960465 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-acc88ee9-c7e3-49bc-b99e-1b1855fae733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:26:59.969559 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 774/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 18:26:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:00.169202 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2e170c-1655-4843-8cf0-95ea52fce9cf None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:00.200945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2e170c-1655-4843-8cf0-95ea52fce9cf None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:00.212025 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c0a971-b5bc-4f12-a9c3-33cc0ba6cbc1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:00.213187 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0a971-b5bc-4f12-a9c3-33cc0ba6cbc1 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:00.213719 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0a971-b5bc-4f12-a9c3-33cc0ba6cbc1 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:00.215457 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0a971-b5bc-4f12-a9c3-33cc0ba6cbc1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:00.233891 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c0a971-b5bc-4f12-a9c3-33cc0ba6cbc1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:00.237889 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c0a971-b5bc-4f12-a9c3-33cc0ba6cbc1 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:00.260919 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 753/1527] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:27:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:27:04.756781 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1a559-a74c-4cf8-b071-406e9c93b0da None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:04.757442 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1a559-a74c-4cf8-b071-406e9c93b0da None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:04.757818 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1a559-a74c-4cf8-b071-406e9c93b0da None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:04.760538 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 775/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 18:27:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:27:04.768025 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb88d4a-c382-4070-b39c-064917c2edfd None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:04.768831 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb88d4a-c382-4070-b39c-064917c2edfd None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:04.768831 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb88d4a-c382-4070-b39c-064917c2edfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:04.802515 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-bcb88d4a-c382-4070-b39c-064917c2edfd None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:04.804679 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-bcb88d4a-c382-4070-b39c-064917c2edfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:04.820502 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 754/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 18:27:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:04.879437 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad1a559-a74c-4cf8-b071-406e9c93b0da None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:04.882409 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3ad1a559-a74c-4cf8-b071-406e9c93b0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:04.991459 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad1a559-a74c-4cf8-b071-406e9c93b0da None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:04.995960 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4507233-69d9-4e2d-964f-61f860116f12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:04.996655 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4507233-69d9-4e2d-964f-61f860116f12 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:04.996937 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4507233-69d9-4e2d-964f-61f860116f12 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:04.997214 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4507233-69d9-4e2d-964f-61f860116f12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:05.009887 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4507233-69d9-4e2d-964f-61f860116f12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:05.011298 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4507233-69d9-4e2d-964f-61f860116f12 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:05.028331 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 776/1530] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:27:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:27:07.555551 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe38de4-3878-472e-b83c-eb54286db7d8 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:07.556378 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe38de4-3878-472e-b83c-eb54286db7d8 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:07.556378 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe38de4-3878-472e-b83c-eb54286db7d8 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:07.557789 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 755/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 18:27:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:27:07.565572 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e70982-4d16-497a-8ae8-6c28256e2076 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:07.566389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e70982-4d16-497a-8ae8-6c28256e2076 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:07.566581 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e70982-4d16-497a-8ae8-6c28256e2076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:07.574272 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=72 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:07.593573 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-77e70982-4d16-497a-8ae8-6c28256e2076 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:07.595350 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-77e70982-4d16-497a-8ae8-6c28256e2076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:07.605624 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 777/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 18:27:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:27:07.647280 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe38de4-3878-472e-b83c-eb54286db7d8 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:07.664581 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe38de4-3878-472e-b83c-eb54286db7d8 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:07.671014 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e7cd5-ff11-4f4a-bebf-044f4764e53a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:07.672146 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e7cd5-ff11-4f4a-bebf-044f4764e53a service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:07.672651 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e7cd5-ff11-4f4a-bebf-044f4764e53a service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:07.673162 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e7cd5-ff11-4f4a-bebf-044f4764e53a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:07.694441 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2e7cd5-ff11-4f4a-bebf-044f4764e53a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:07.697271 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2e7cd5-ff11-4f4a-bebf-044f4764e53a service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:07.717456 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 756/1533] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:27:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:27:12.440102 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4d4aa-ec55-46f9-bcd2-0af20c38ebd5 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:12.440923 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4d4aa-ec55-46f9-bcd2-0af20c38ebd5 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:12.440923 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4d4aa-ec55-46f9-bcd2-0af20c38ebd5 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:12.443682 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 778/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 18:27:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:27:12.451460 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1f874-2d12-47b4-a0d2-a10b6def7001 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:12.452107 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1f874-2d12-47b4-a0d2-a10b6def7001 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:12.452107 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1f874-2d12-47b4-a0d2-a10b6def7001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:12.475660 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-0ad1f874-2d12-47b4-a0d2-a10b6def7001 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:12.477422 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0ad1f874-2d12-47b4-a0d2-a10b6def7001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:12.487801 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 757/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 18:27:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:12.845521 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e4d4aa-ec55-46f9-bcd2-0af20c38ebd5 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:12.848038 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f7e4d4aa-ec55-46f9-bcd2-0af20c38ebd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:12.945189 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e4d4aa-ec55-46f9-bcd2-0af20c38ebd5 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:12.951873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24875daa-e16d-4939-ae63-e089621dd6b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:12.953128 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24875daa-e16d-4939-ae63-e089621dd6b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:12.953628 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24875daa-e16d-4939-ae63-e089621dd6b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:12.954168 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24875daa-e16d-4939-ae63-e089621dd6b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:12.971519 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24875daa-e16d-4939-ae63-e089621dd6b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:12.973734 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24875daa-e16d-4939-ae63-e089621dd6b8 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:13.002866 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 779/1536] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:27:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:27:15.456667 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28085837-3bad-4872-954d-df1ec0443abe None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:15.457505 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28085837-3bad-4872-954d-df1ec0443abe None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:15.457707 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28085837-3bad-4872-954d-df1ec0443abe None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:15.459840 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 758/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 18:27:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:27:15.466980 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333a73e-b775-4b5c-b7a4-d0866a06a05d None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:15.467585 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333a73e-b775-4b5c-b7a4-d0866a06a05d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:15.467737 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333a73e-b775-4b5c-b7a4-d0866a06a05d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:15.497671 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-2333a73e-b775-4b5c-b7a4-d0866a06a05d None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:15.499580 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-2333a73e-b775-4b5c-b7a4-d0866a06a05d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:15.509236 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 780/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 18:27:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:15.923777 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28085837-3bad-4872-954d-df1ec0443abe None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:15.927414 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-28085837-3bad-4872-954d-df1ec0443abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:15.940414 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=94 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:15.950948 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28085837-3bad-4872-954d-df1ec0443abe None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:15.955848 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe2f4db-16f6-43a2-87db-50b798ef7fe1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:15.956602 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe2f4db-16f6-43a2-87db-50b798ef7fe1 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:15.956970 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe2f4db-16f6-43a2-87db-50b798ef7fe1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:15.957250 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe2f4db-16f6-43a2-87db-50b798ef7fe1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:15.976800 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe2f4db-16f6-43a2-87db-50b798ef7fe1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:15.979871 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe2f4db-16f6-43a2-87db-50b798ef7fe1 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:15.999138 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 759/1539] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:27:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:27:25.488618 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=31 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:25.990986 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:03.492819 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b16318d-7fc2-4f37-bab5-624fbba495f6 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:03.496211 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b16318d-7fc2-4f37-bab5-624fbba495f6 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:03.496211 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b16318d-7fc2-4f37-bab5-624fbba495f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:03.531147 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-0b16318d-7fc2-4f37-bab5-624fbba495f6 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:03.533298 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-0b16318d-7fc2-4f37-bab5-624fbba495f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:03.555150 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 781/1540] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:03.576716 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac636fd-9de6-4fc6-a9fd-59156849432b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:03.578883 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac636fd-9de6-4fc6-a9fd-59156849432b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:03.580300 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac636fd-9de6-4fc6-a9fd-59156849432b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:03.623160 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-1ac636fd-9de6-4fc6-a9fd-59156849432b None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:03.626325 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1ac636fd-9de6-4fc6-a9fd-59156849432b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:03.644667 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3779c8-40b1-412b-9e97-fc82ab9c594f None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:03.645152 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3779c8-40b1-412b-9e97-fc82ab9c594f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:03.645695 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3779c8-40b1-412b-9e97-fc82ab9c594f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:03.658796 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 760/1541] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:03.667846 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac636fd-9de6-4fc6-a9fd-59156849432b None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:03.695824 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac636fd-9de6-4fc6-a9fd-59156849432b None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:03.703485 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-9d3779c8-40b1-412b-9e97-fc82ab9c594f None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:03.705239 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-9d3779c8-40b1-412b-9e97-fc82ab9c594f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:03.711000 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dbb480-1427-4cb7-955b-df5a38396e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:03.712671 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dbb480-1427-4cb7-955b-df5a38396e28 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:03.713464 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dbb480-1427-4cb7-955b-df5a38396e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:03.714314 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dbb480-1427-4cb7-955b-df5a38396e28 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:03.715978 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dbb480-1427-4cb7-955b-df5a38396e28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:29:03.722035 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dbb480-1427-4cb7-955b-df5a38396e28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:03.724927 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: WARNING py.warnings [None req-16dbb480-1427-4cb7-955b-df5a38396e28 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 21 18:29:03.724927 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: warnings.warn(msg) Jun 21 18:29:03.724927 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: Jun 21 18:29:03.728541 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dbb480-1427-4cb7-955b-df5a38396e28 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:03.738553 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 782/1542] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:03.764169 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 761/1543] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:29:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:29:03.767615 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2cb958-b17c-4913-807d-b52cdad46c14 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:03.768125 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2cb958-b17c-4913-807d-b52cdad46c14 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:03.768620 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2cb958-b17c-4913-807d-b52cdad46c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:03.799236 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dbb480-1427-4cb7-955b-df5a38396e28 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:03.832859 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dbb480-1427-4cb7-955b-df5a38396e28 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:03.852238 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841e39ce-c6fd-49b8-b5ab-dfe38d97865e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:03.852628 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-5b2cb958-b17c-4913-807d-b52cdad46c14 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:03.854354 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841e39ce-c6fd-49b8-b5ab-dfe38d97865e admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:03.854947 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841e39ce-c6fd-49b8-b5ab-dfe38d97865e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:03.855475 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841e39ce-c6fd-49b8-b5ab-dfe38d97865e admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:03.856470 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841e39ce-c6fd-49b8-b5ab-dfe38d97865e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:03.859277 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5b2cb958-b17c-4913-807d-b52cdad46c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:03.861484 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841e39ce-c6fd-49b8-b5ab-dfe38d97865e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:03.890942 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 783/1544] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:03.899667 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2cb958-b17c-4913-807d-b52cdad46c14 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:03.922942 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2cb958-b17c-4913-807d-b52cdad46c14 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:03.936421 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd66d30-d387-42c4-8441-616f2bba478c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:03.937741 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd66d30-d387-42c4-8441-616f2bba478c admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:03.938378 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd66d30-d387-42c4-8441-616f2bba478c admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:03.938916 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd66d30-d387-42c4-8441-616f2bba478c admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:03.940843 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd66d30-d387-42c4-8441-616f2bba478c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:29:03.941441 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd66d30-d387-42c4-8441-616f2bba478c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:03.942063 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 762/1545] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:29:03 2022] POST /identity/v3/projects => generated 391 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:29:03.952124 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841e39ce-c6fd-49b8-b5ab-dfe38d97865e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:03.953507 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd66d30-d387-42c4-8441-616f2bba478c admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:03.978120 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 784/1546] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:29:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:29:03.994450 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd66d30-d387-42c4-8441-616f2bba478c admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:04.019379 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd66d30-d387-42c4-8441-616f2bba478c admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:04.172688 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c37b18a-7f7e-4b9b-9a30-1f90b03491f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:04.179275 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37b18a-7f7e-4b9b-9a30-1f90b03491f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:04.179879 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37b18a-7f7e-4b9b-9a30-1f90b03491f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:04.180512 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37b18a-7f7e-4b9b-9a30-1f90b03491f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:04.184745 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c37b18a-7f7e-4b9b-9a30-1f90b03491f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:04.187114 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: WARNING py.warnings [None req-3c37b18a-7f7e-4b9b-9a30-1f90b03491f4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 18:29:04.187114 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: warnings.warn(msg) Jun 21 18:29:04.187114 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: Jun 21 18:29:04.193118 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841e39ce-c6fd-49b8-b5ab-dfe38d97865e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:04.201681 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c37b18a-7f7e-4b9b-9a30-1f90b03491f4 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:04.234848 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f428b1d9-f2b7-44ef-ac0c-acb93acea472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:04.236359 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f428b1d9-f2b7-44ef-ac0c-acb93acea472 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:04.237026 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f428b1d9-f2b7-44ef-ac0c-acb93acea472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:04.237708 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f428b1d9-f2b7-44ef-ac0c-acb93acea472 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:04.242342 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f428b1d9-f2b7-44ef-ac0c-acb93acea472 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:04.244861 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: WARNING py.warnings [None req-f428b1d9-f2b7-44ef-ac0c-acb93acea472 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 18:29:04.244861 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: warnings.warn(msg) Jun 21 18:29:04.244861 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: Jun 21 18:29:04.252524 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f428b1d9-f2b7-44ef-ac0c-acb93acea472 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:04.308547 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 763/1547] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:29:03 2022] POST /identity/v3/users => generated 296 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:04.311855 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07476ad-f57a-48bb-97dc-ed5e103697c2 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:04.314988 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07476ad-f57a-48bb-97dc-ed5e103697c2 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:04.321570 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07476ad-f57a-48bb-97dc-ed5e103697c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:04.325494 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 785/1548] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:29:03 2022] POST /identity/v3/projects => generated 379 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:04.328185 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542d4b7-20cd-4a3c-ae4d-2b100b39b4e8 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:04.328642 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542d4b7-20cd-4a3c-ae4d-2b100b39b4e8 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:04.329154 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542d4b7-20cd-4a3c-ae4d-2b100b39b4e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:04.357607 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b07476ad-f57a-48bb-97dc-ed5e103697c2 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:04.361463 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b07476ad-f57a-48bb-97dc-ed5e103697c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:04.403974 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-a542d4b7-20cd-4a3c-ae4d-2b100b39b4e8 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:04.415289 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a542d4b7-20cd-4a3c-ae4d-2b100b39b4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:04.522602 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 764/1549] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:04.527146 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e7229-847d-4638-b72d-e46028678c7f None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:04.528099 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e7229-847d-4638-b72d-e46028678c7f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:04.529118 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e7229-847d-4638-b72d-e46028678c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:04.560713 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 786/1550] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:04.561941 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542d4b7-20cd-4a3c-ae4d-2b100b39b4e8 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:04.575826 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-cc9e7229-847d-4638-b72d-e46028678c7f None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:04.578081 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-cc9e7229-847d-4638-b72d-e46028678c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:04.579263 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542d4b7-20cd-4a3c-ae4d-2b100b39b4e8 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:04.585582 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8121dc8-1e92-4310-8c83-5d08155ea44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:04.586439 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8121dc8-1e92-4310-8c83-5d08155ea44e admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:04.586786 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8121dc8-1e92-4310-8c83-5d08155ea44e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:04.587217 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8121dc8-1e92-4310-8c83-5d08155ea44e admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:04.588495 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8121dc8-1e92-4310-8c83-5d08155ea44e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:04.591160 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 765/1551] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:04.591595 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8121dc8-1e92-4310-8c83-5d08155ea44e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:04.591973 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e7229-847d-4638-b72d-e46028678c7f None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:04.606269 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e7229-847d-4638-b72d-e46028678c7f None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:04.611177 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 787/1552] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:04.614500 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada8bf7-edc4-4631-b119-3fee5b302577 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:04.615089 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77496b3c-1d83-4d0f-b5a5-0c4818495264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:04.615296 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada8bf7-edc4-4631-b119-3fee5b302577 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:04.615592 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada8bf7-edc4-4631-b119-3fee5b302577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:04.615958 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77496b3c-1d83-4d0f-b5a5-0c4818495264 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:04.616256 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77496b3c-1d83-4d0f-b5a5-0c4818495264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:04.616545 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77496b3c-1d83-4d0f-b5a5-0c4818495264 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:04.617332 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77496b3c-1d83-4d0f-b5a5-0c4818495264 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:04.619801 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77496b3c-1d83-4d0f-b5a5-0c4818495264 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:04.644631 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-9ada8bf7-edc4-4631-b119-3fee5b302577 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:04.646571 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-9ada8bf7-edc4-4631-b119-3fee5b302577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:04.650144 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 766/1553] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:29:04 2022] POST /identity/v3/users => generated 290 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:04.651789 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9402196-7467-46b4-979e-6343deea2170 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:04.652091 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9402196-7467-46b4-979e-6343deea2170 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:04.652446 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9402196-7467-46b4-979e-6343deea2170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:04.660703 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 788/1554] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:04.662819 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd310a68-4317-4f0a-8cd4-27e2d05e5bf3 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:04.663252 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd310a68-4317-4f0a-8cd4-27e2d05e5bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:04.663664 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd310a68-4317-4f0a-8cd4-27e2d05e5bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:04.684781 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f9402196-7467-46b4-979e-6343deea2170 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:04.686755 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f9402196-7467-46b4-979e-6343deea2170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:04.692154 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-bd310a68-4317-4f0a-8cd4-27e2d05e5bf3 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:04.693725 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-bd310a68-4317-4f0a-8cd4-27e2d05e5bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:04.700620 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 767/1555] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:04.703615 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77977b02-7a70-4fe7-b470-c8411a5a5911 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:04.704116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77977b02-7a70-4fe7-b470-c8411a5a5911 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:04.704759 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77977b02-7a70-4fe7-b470-c8411a5a5911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:04.712578 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 789/1556] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:04.712578 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ec827-9d81-48dd-9bc4-31a453445a74 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:04.712578 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ec827-9d81-48dd-9bc4-31a453445a74 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:04.712578 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ec827-9d81-48dd-9bc4-31a453445a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:04.738233 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-2c1ec827-9d81-48dd-9bc4-31a453445a74 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:04.739705 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-2c1ec827-9d81-48dd-9bc4-31a453445a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:04.745419 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-77977b02-7a70-4fe7-b470-c8411a5a5911 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:04.747080 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-77977b02-7a70-4fe7-b470-c8411a5a5911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:04.749886 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 790/1557] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:04.751554 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86150b97-099d-468f-9161-23c8efc0d736 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:04.751956 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86150b97-099d-468f-9161-23c8efc0d736 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:04.752296 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86150b97-099d-468f-9161-23c8efc0d736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:04.759920 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 768/1558] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:04.761178 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77977b02-7a70-4fe7-b470-c8411a5a5911 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:04.777940 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-86150b97-099d-468f-9161-23c8efc0d736 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:04.778261 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77977b02-7a70-4fe7-b470-c8411a5a5911 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:04.779387 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-86150b97-099d-468f-9161-23c8efc0d736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:04.787216 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3292f8ab-3b83-4e31-9df8-16eb67ddb6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:04.788408 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3292f8ab-3b83-4e31-9df8-16eb67ddb6e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:04.788909 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3292f8ab-3b83-4e31-9df8-16eb67ddb6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:04.789359 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3292f8ab-3b83-4e31-9df8-16eb67ddb6e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:04.790353 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 791/1559] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:04.790852 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3292f8ab-3b83-4e31-9df8-16eb67ddb6e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:04.791405 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86150b97-099d-468f-9161-23c8efc0d736 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:04.794521 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3292f8ab-3b83-4e31-9df8-16eb67ddb6e4 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:04.803823 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86150b97-099d-468f-9161-23c8efc0d736 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:04.806037 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:04.816433 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe819798-ab68-4f17-b691-9ddfb37277a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:04.822863 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 769/1560] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:04.823141 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe819798-ab68-4f17-b691-9ddfb37277a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:04.823141 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe819798-ab68-4f17-b691-9ddfb37277a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:04.823141 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe819798-ab68-4f17-b691-9ddfb37277a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:04.823141 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe819798-ab68-4f17-b691-9ddfb37277a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:04.823773 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c306fcb-861f-43d7-85ab-f1d4f1b93145 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:04.823773 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c306fcb-861f-43d7-85ab-f1d4f1b93145 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:04.823773 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c306fcb-861f-43d7-85ab-f1d4f1b93145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:04.824197 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe819798-ab68-4f17-b691-9ddfb37277a7 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:04.843335 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 792/1561] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:04.843335 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe819798-ab68-4f17-b691-9ddfb37277a7 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:04.853380 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:04.856284 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-5c306fcb-861f-43d7-85ab-f1d4f1b93145 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:04.858562 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5c306fcb-861f-43d7-85ab-f1d4f1b93145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:04.860041 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe819798-ab68-4f17-b691-9ddfb37277a7 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:04.869770 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b708a849-29c2-4bcd-a7b2-25b68a5613f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:04.870301 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708a849-29c2-4bcd-a7b2-25b68a5613f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:04.870723 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708a849-29c2-4bcd-a7b2-25b68a5613f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:04.870900 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708a849-29c2-4bcd-a7b2-25b68a5613f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:04.871807 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b708a849-29c2-4bcd-a7b2-25b68a5613f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:29:04.872224 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b708a849-29c2-4bcd-a7b2-25b68a5613f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:04.874474 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 770/1562] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:04.874709 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b708a849-29c2-4bcd-a7b2-25b68a5613f6 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:04.876660 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a6df42-2bfa-484a-97be-8cf46f5df4da None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:04.877321 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a6df42-2bfa-484a-97be-8cf46f5df4da None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:04.877853 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a6df42-2bfa-484a-97be-8cf46f5df4da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:04.891993 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 793/1563] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:29:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:29:04.893832 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b708a849-29c2-4bcd-a7b2-25b68a5613f6 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:04.906938 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b708a849-29c2-4bcd-a7b2-25b68a5613f6 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:04.908341 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-27a6df42-2bfa-484a-97be-8cf46f5df4da None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:04.910576 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-27a6df42-2bfa-484a-97be-8cf46f5df4da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:04.913195 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83d4cb1-15e6-49fd-881f-0a03202d9e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:04.913972 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d4cb1-15e6-49fd-881f-0a03202d9e50 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:04.914275 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d4cb1-15e6-49fd-881f-0a03202d9e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:04.914547 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d4cb1-15e6-49fd-881f-0a03202d9e50 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:04.915306 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83d4cb1-15e6-49fd-881f-0a03202d9e50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:29:04.915631 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83d4cb1-15e6-49fd-881f-0a03202d9e50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:04.917479 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83d4cb1-15e6-49fd-881f-0a03202d9e50 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:04.925587 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 771/1564] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:04.927777 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e006e30-0eec-435d-98fc-78978c93baa4 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:04.928227 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e006e30-0eec-435d-98fc-78978c93baa4 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:04.928653 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e006e30-0eec-435d-98fc-78978c93baa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:04.936763 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 794/1565] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:29:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:29:04.938189 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83d4cb1-15e6-49fd-881f-0a03202d9e50 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:04.951728 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83d4cb1-15e6-49fd-881f-0a03202d9e50 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:04.963290 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb412537-6fcd-41e5-bb67-76aedc6c4098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:04.964194 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-2e006e30-0eec-435d-98fc-78978c93baa4 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:04.964406 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb412537-6fcd-41e5-bb67-76aedc6c4098 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:04.964809 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb412537-6fcd-41e5-bb67-76aedc6c4098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:04.965275 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb412537-6fcd-41e5-bb67-76aedc6c4098 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:04.965717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2e006e30-0eec-435d-98fc-78978c93baa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:04.966405 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb412537-6fcd-41e5-bb67-76aedc6c4098 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:29:04.966913 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb412537-6fcd-41e5-bb67-76aedc6c4098 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:04.974516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb412537-6fcd-41e5-bb67-76aedc6c4098 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:04.981109 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 772/1566] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:04.985062 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e006e30-0eec-435d-98fc-78978c93baa4 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:04.997364 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 795/1567] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:29:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:29:04.998611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb412537-6fcd-41e5-bb67-76aedc6c4098 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:05.004298 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e006e30-0eec-435d-98fc-78978c93baa4 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:05.013976 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb412537-6fcd-41e5-bb67-76aedc6c4098 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:05.014300 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77e9c75-8b9b-4be0-b4f0-d0fc7814a500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:05.015768 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e9c75-8b9b-4be0-b4f0-d0fc7814a500 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:05.016415 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e9c75-8b9b-4be0-b4f0-d0fc7814a500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:05.016874 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e9c75-8b9b-4be0-b4f0-d0fc7814a500 admin admin] PATH_INFO: `/v3/projects/6538654ef94d4036a7a8d5614b80b0ed/users/de09b91800314d06aeddafe3c12a77b3/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:05.020672 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b64bcaf-8cc1-4b5f-b278-8148819c28cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:05.021674 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b64bcaf-8cc1-4b5f-b278-8148819c28cd admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:05.022075 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b64bcaf-8cc1-4b5f-b278-8148819c28cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:05.022561 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b64bcaf-8cc1-4b5f-b278-8148819c28cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:05.023931 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b64bcaf-8cc1-4b5f-b278-8148819c28cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:05.026380 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b64bcaf-8cc1-4b5f-b278-8148819c28cd admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:05.045097 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77e9c75-8b9b-4be0-b4f0-d0fc7814a500 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6538654ef94d4036a7a8d5614b80b0ed, user_id=de09b91800314d06aeddafe3c12a77b3, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:05.047648 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 796/1568] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:05.048356 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77e9c75-8b9b-4be0-b4f0-d0fc7814a500 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:05.049026 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b64bcaf-8cc1-4b5f-b278-8148819c28cd admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:05.069547 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b64bcaf-8cc1-4b5f-b278-8148819c28cd admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:05.076180 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 773/1569] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:29:04 2022] PUT /identity/v3/projects/6538654ef94d4036a7a8d5614b80b0ed/users/de09b91800314d06aeddafe3c12a77b3/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:05.077679 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77e9c75-8b9b-4be0-b4f0-d0fc7814a500 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:05.185739 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d601fa4-d854-402f-9d91-ac5c4f44d7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:05.186894 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d601fa4-d854-402f-9d91-ac5c4f44d7f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:05.187641 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d601fa4-d854-402f-9d91-ac5c4f44d7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:05.187977 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d601fa4-d854-402f-9d91-ac5c4f44d7f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:05.189457 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d601fa4-d854-402f-9d91-ac5c4f44d7f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:29:05.190060 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d601fa4-d854-402f-9d91-ac5c4f44d7f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:05.193367 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d601fa4-d854-402f-9d91-ac5c4f44d7f0 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:05.200474 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77e9c75-8b9b-4be0-b4f0-d0fc7814a500 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:05.208374 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166ff2b2-b779-49d3-a941-677b157b05aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:05.209499 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ff2b2-b779-49d3-a941-677b157b05aa admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:05.210023 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ff2b2-b779-49d3-a941-677b157b05aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:05.210517 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ff2b2-b779-49d3-a941-677b157b05aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:05.211790 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166ff2b2-b779-49d3-a941-677b157b05aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:05.225774 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 797/1570] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:29:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:29:05.225774 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d601fa4-d854-402f-9d91-ac5c4f44d7f0 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:05.226319 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166ff2b2-b779-49d3-a941-677b157b05aa admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:05.246976 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d601fa4-d854-402f-9d91-ac5c4f44d7f0 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:05.255454 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abab470-3503-4986-b02d-e69f3e01e423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:05.256363 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abab470-3503-4986-b02d-e69f3e01e423 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:05.256789 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abab470-3503-4986-b02d-e69f3e01e423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:05.257145 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abab470-3503-4986-b02d-e69f3e01e423 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:05.258302 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abab470-3503-4986-b02d-e69f3e01e423 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:29:05.258692 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abab470-3503-4986-b02d-e69f3e01e423 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:05.260952 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abab470-3503-4986-b02d-e69f3e01e423 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:05.267324 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 774/1571] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:29:05 2022] POST /identity/v3/projects => generated 405 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:05.269254 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166ff2b2-b779-49d3-a941-677b157b05aa admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:05.281032 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 798/1572] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:29:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:29:05.282346 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abab470-3503-4986-b02d-e69f3e01e423 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:05.294971 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abab470-3503-4986-b02d-e69f3e01e423 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:05.369541 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166ff2b2-b779-49d3-a941-677b157b05aa admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:05.374709 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4151226-f99b-4195-a031-c911d110b576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:05.375400 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4151226-f99b-4195-a031-c911d110b576 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:05.375858 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4151226-f99b-4195-a031-c911d110b576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:05.376177 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4151226-f99b-4195-a031-c911d110b576 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:05.377190 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4151226-f99b-4195-a031-c911d110b576 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:05.379659 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4151226-f99b-4195-a031-c911d110b576 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:05.387488 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc566a-c74a-486e-9890-2dc10b1d6a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:05.388238 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc566a-c74a-486e-9890-2dc10b1d6a5d admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:05.388561 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc566a-c74a-486e-9890-2dc10b1d6a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:05.388872 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc566a-c74a-486e-9890-2dc10b1d6a5d admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:05.393727 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfc566a-c74a-486e-9890-2dc10b1d6a5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:29:05.394123 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfc566a-c74a-486e-9890-2dc10b1d6a5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:05.396193 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfc566a-c74a-486e-9890-2dc10b1d6a5d admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:05.413276 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 799/1573] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:29:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:29:05.414231 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc566a-c74a-486e-9890-2dc10b1d6a5d admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:05.428745 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 775/1574] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:29:05 2022] POST /identity/v3/projects => generated 365 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:05.430233 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4151226-f99b-4195-a031-c911d110b576 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:05.443165 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc566a-c74a-486e-9890-2dc10b1d6a5d admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:05.472283 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:05.490749 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:05.521407 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4151226-f99b-4195-a031-c911d110b576 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:05.527442 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb8e9cb-1339-4d45-9c93-50e34a20e9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:05.528566 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8e9cb-1339-4d45-9c93-50e34a20e9f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:05.528945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8e9cb-1339-4d45-9c93-50e34a20e9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:05.529297 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8e9cb-1339-4d45-9c93-50e34a20e9f1 admin admin] PATH_INFO: `/v3/projects/ad7bfb54384a4d648245aaf12c05ade7/users/a03657e872e94266b12d95a602038cda/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:05.539077 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bf1f57-ce8b-436c-99fa-177390352275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:05.539977 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf1f57-ce8b-436c-99fa-177390352275 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:05.540285 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf1f57-ce8b-436c-99fa-177390352275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:05.540621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf1f57-ce8b-436c-99fa-177390352275 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:05.541545 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bf1f57-ce8b-436c-99fa-177390352275 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:05.544151 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bf1f57-ce8b-436c-99fa-177390352275 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:05.549568 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb8e9cb-1339-4d45-9c93-50e34a20e9f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad7bfb54384a4d648245aaf12c05ade7, user_id=a03657e872e94266b12d95a602038cda, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:05.554859 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb8e9cb-1339-4d45-9c93-50e34a20e9f1 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:05.575217 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 776/1575] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:29:05 2022] PUT /identity/v3/projects/ad7bfb54384a4d648245aaf12c05ade7/users/a03657e872e94266b12d95a602038cda/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:05.577411 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c8655-6da1-4a2f-bae9-0c143bfa824b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:05.578187 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c8655-6da1-4a2f-bae9-0c143bfa824b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:05.579157 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c8655-6da1-4a2f-bae9-0c143bfa824b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:05.589140 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 800/1576] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:29:05 2022] POST /identity/v3/projects => generated 385 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:05.589140 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bf1f57-ce8b-436c-99fa-177390352275 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:05.606421 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-805c8655-6da1-4a2f-bae9-0c143bfa824b None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:05.608309 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-805c8655-6da1-4a2f-bae9-0c143bfa824b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:05.674164 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bf1f57-ce8b-436c-99fa-177390352275 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:05.682477 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc3e08c-9f09-4a7a-9f3b-8411d1e93e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:05.683593 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3e08c-9f09-4a7a-9f3b-8411d1e93e21 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:05.684079 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3e08c-9f09-4a7a-9f3b-8411d1e93e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:05.684487 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3e08c-9f09-4a7a-9f3b-8411d1e93e21 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:05.685855 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc3e08c-9f09-4a7a-9f3b-8411d1e93e21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:05.689960 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc3e08c-9f09-4a7a-9f3b-8411d1e93e21 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:05.702768 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 777/1577] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:05.704329 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c8655-6da1-4a2f-bae9-0c143bfa824b None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:05.723192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c8655-6da1-4a2f-bae9-0c143bfa824b None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:05.731159 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006bc2e2-e929-486c-9f9e-6a4801fcd0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:05.732195 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 801/1578] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:29:05 2022] POST /identity/v3/projects => generated 377 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:05.732489 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006bc2e2-e929-486c-9f9e-6a4801fcd0d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:05.733009 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006bc2e2-e929-486c-9f9e-6a4801fcd0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:05.733209 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc3e08c-9f09-4a7a-9f3b-8411d1e93e21 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:05.734251 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006bc2e2-e929-486c-9f9e-6a4801fcd0d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:05.735591 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006bc2e2-e929-486c-9f9e-6a4801fcd0d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:05.740608 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006bc2e2-e929-486c-9f9e-6a4801fcd0d7 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:05.788311 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 778/1579] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:29:05 2022] POST /identity/v3/users => generated 303 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:05.789435 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006bc2e2-e929-486c-9f9e-6a4801fcd0d7 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:05.801574 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006bc2e2-e929-486c-9f9e-6a4801fcd0d7 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:05.840832 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc3e08c-9f09-4a7a-9f3b-8411d1e93e21 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:05.849857 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eaff0e-03e4-4f48-8105-dbe27b009655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:05.851036 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eaff0e-03e4-4f48-8105-dbe27b009655 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:05.851495 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eaff0e-03e4-4f48-8105-dbe27b009655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:05.852038 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eaff0e-03e4-4f48-8105-dbe27b009655 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:05.853267 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eaff0e-03e4-4f48-8105-dbe27b009655 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:05.857541 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eaff0e-03e4-4f48-8105-dbe27b009655 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:05.889165 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0cc6fd-8cfa-49bb-836a-385402c6e5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:05.890110 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0cc6fd-8cfa-49bb-836a-385402c6e5aa admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:05.890568 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0cc6fd-8cfa-49bb-836a-385402c6e5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:05.891007 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0cc6fd-8cfa-49bb-836a-385402c6e5aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:05.891911 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0cc6fd-8cfa-49bb-836a-385402c6e5aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:05.894699 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0cc6fd-8cfa-49bb-836a-385402c6e5aa admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:05.901896 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 802/1580] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:29:05 2022] POST /identity/v3/projects => generated 367 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:05.902771 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eaff0e-03e4-4f48-8105-dbe27b009655 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:05.937300 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 779/1581] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:29:05 2022] POST /identity/v3/projects => generated 383 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:05.939909 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43181de-f366-49ba-9c5f-70e7a757ace6 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:05.940432 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43181de-f366-49ba-9c5f-70e7a757ace6 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:05.940904 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43181de-f366-49ba-9c5f-70e7a757ace6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:05.943859 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:05.971740 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-c43181de-f366-49ba-9c5f-70e7a757ace6 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:05.973895 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c43181de-f366-49ba-9c5f-70e7a757ace6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:05.990592 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eaff0e-03e4-4f48-8105-dbe27b009655 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:05.998398 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:06.062962 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb3fb4-e9a2-4ca8-808d-06f6e1d00ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:06.063693 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb3fb4-e9a2-4ca8-808d-06f6e1d00ff6 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:06.064127 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb3fb4-e9a2-4ca8-808d-06f6e1d00ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:06.064732 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb3fb4-e9a2-4ca8-808d-06f6e1d00ff6 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:06.065502 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbb3fb4-e9a2-4ca8-808d-06f6e1d00ff6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:06.067791 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbb3fb4-e9a2-4ca8-808d-06f6e1d00ff6 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:06.082863 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 780/1582] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:06.084311 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43181de-f366-49ba-9c5f-70e7a757ace6 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:06.098374 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 803/1583] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:29:05 2022] POST /identity/v3/users => generated 283 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:06.100466 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb3fb4-e9a2-4ca8-808d-06f6e1d00ff6 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:06.102482 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43181de-f366-49ba-9c5f-70e7a757ace6 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:06.110174 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f4efa-273a-4921-b683-f98e4458adf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:06.111072 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f4efa-273a-4921-b683-f98e4458adf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:06.111459 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f4efa-273a-4921-b683-f98e4458adf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:06.111830 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f4efa-273a-4921-b683-f98e4458adf7 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:06.112963 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26f4efa-273a-4921-b683-f98e4458adf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:06.115302 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26f4efa-273a-4921-b683-f98e4458adf7 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:06.141393 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 781/1584] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:29:06 2022] POST /identity/v3/users => generated 293 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:06.143111 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f4efa-273a-4921-b683-f98e4458adf7 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:06.156258 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f4efa-273a-4921-b683-f98e4458adf7 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:06.164228 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53e4672-96b1-4568-b4c5-c9a7fdafaedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:06.165728 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e4672-96b1-4568-b4c5-c9a7fdafaedc admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:06.166189 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e4672-96b1-4568-b4c5-c9a7fdafaedc admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:06.166697 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e4672-96b1-4568-b4c5-c9a7fdafaedc admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:06.167661 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53e4672-96b1-4568-b4c5-c9a7fdafaedc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:06.170229 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53e4672-96b1-4568-b4c5-c9a7fdafaedc admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:06.194915 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb3fb4-e9a2-4ca8-808d-06f6e1d00ff6 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:06.199815 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e5ab9-5a9a-4084-b472-a0be1898d130 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:06.200539 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e5ab9-5a9a-4084-b472-a0be1898d130 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:06.200854 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e5ab9-5a9a-4084-b472-a0be1898d130 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:06.201165 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e5ab9-5a9a-4084-b472-a0be1898d130 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:06.202747 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 782/1585] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:29:06 2022] POST /identity/v3/users => generated 289 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:06.203763 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53e4672-96b1-4568-b4c5-c9a7fdafaedc admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:06.216688 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53e4672-96b1-4568-b4c5-c9a7fdafaedc admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:06.219770 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48e5ab9-5a9a-4084-b472-a0be1898d130 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:06.222189 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48e5ab9-5a9a-4084-b472-a0be1898d130 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:06.223465 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a94da1f-87d6-438c-bee8-77496cec7792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:06.224598 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a94da1f-87d6-438c-bee8-77496cec7792 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:06.225067 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a94da1f-87d6-438c-bee8-77496cec7792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:06.225535 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a94da1f-87d6-438c-bee8-77496cec7792 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:06.226829 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a94da1f-87d6-438c-bee8-77496cec7792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:06.229788 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a94da1f-87d6-438c-bee8-77496cec7792 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:06.238286 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 804/1586] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:29:06.243368 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e5ab9-5a9a-4084-b472-a0be1898d130 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:06.248059 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 783/1587] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:06.249176 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a94da1f-87d6-438c-bee8-77496cec7792 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:06.262405 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a94da1f-87d6-438c-bee8-77496cec7792 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:06.265282 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e5ab9-5a9a-4084-b472-a0be1898d130 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:06.269017 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c17d52-3ae0-4a52-8a94-252ad3ab0f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:06.270164 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c17d52-3ae0-4a52-8a94-252ad3ab0f45 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:06.270720 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c17d52-3ae0-4a52-8a94-252ad3ab0f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:06.271214 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c17d52-3ae0-4a52-8a94-252ad3ab0f45 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:06.272424 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c17d52-3ae0-4a52-8a94-252ad3ab0f45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:06.272622 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4060ed14-fc75-41be-8705-64c98cceba2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:06.273206 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4060ed14-fc75-41be-8705-64c98cceba2c admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:06.273479 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4060ed14-fc75-41be-8705-64c98cceba2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:06.273801 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4060ed14-fc75-41be-8705-64c98cceba2c admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:06.274506 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4060ed14-fc75-41be-8705-64c98cceba2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:06.276441 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c17d52-3ae0-4a52-8a94-252ad3ab0f45 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:06.276688 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4060ed14-fc75-41be-8705-64c98cceba2c admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:06.303509 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 805/1588] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:29:06 2022] POST /identity/v3/users => generated 284 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:06.304097 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4060ed14-fc75-41be-8705-64c98cceba2c admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:06.315500 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 784/1589] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:29:06 2022] POST /identity/v3/users => generated 292 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:06.315500 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c17d52-3ae0-4a52-8a94-252ad3ab0f45 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:06.321703 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4060ed14-fc75-41be-8705-64c98cceba2c admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:06.326381 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf33517-eaa7-4d8e-8fcb-54bfad5e3747 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:06.327109 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf33517-eaa7-4d8e-8fcb-54bfad5e3747 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:06.327386 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf33517-eaa7-4d8e-8fcb-54bfad5e3747 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:06.327667 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf33517-eaa7-4d8e-8fcb-54bfad5e3747 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:06.332510 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c17d52-3ae0-4a52-8a94-252ad3ab0f45 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:06.338864 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf33517-eaa7-4d8e-8fcb-54bfad5e3747 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:06.340345 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf33517-eaa7-4d8e-8fcb-54bfad5e3747 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:06.341700 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3293dd0-3cc7-4d46-9327-ca31caf4922f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:06.343377 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3293dd0-3cc7-4d46-9327-ca31caf4922f admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:06.343377 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3293dd0-3cc7-4d46-9327-ca31caf4922f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:06.343778 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3293dd0-3cc7-4d46-9327-ca31caf4922f admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:06.345021 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3293dd0-3cc7-4d46-9327-ca31caf4922f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:06.348057 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3293dd0-3cc7-4d46-9327-ca31caf4922f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:06.354759 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 806/1590] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:29:06.355982 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf33517-eaa7-4d8e-8fcb-54bfad5e3747 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:06.368661 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf33517-eaa7-4d8e-8fcb-54bfad5e3747 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:06.370961 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 785/1591] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:06.372625 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3293dd0-3cc7-4d46-9327-ca31caf4922f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:06.375435 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d0f981-e0c8-4719-a2f8-12e84009bc98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:06.376411 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0f981-e0c8-4719-a2f8-12e84009bc98 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:06.376938 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0f981-e0c8-4719-a2f8-12e84009bc98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:06.377459 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0f981-e0c8-4719-a2f8-12e84009bc98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:06.378742 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d0f981-e0c8-4719-a2f8-12e84009bc98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:06.381095 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d0f981-e0c8-4719-a2f8-12e84009bc98 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:06.391535 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3293dd0-3cc7-4d46-9327-ca31caf4922f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:06.400376 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 807/1592] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:06.400988 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1c1e68-b92d-40ce-abda-e77be9c8660b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:06.402073 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d0f981-e0c8-4719-a2f8-12e84009bc98 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:06.402215 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1c1e68-b92d-40ce-abda-e77be9c8660b admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:06.402928 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1c1e68-b92d-40ce-abda-e77be9c8660b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:06.403431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1c1e68-b92d-40ce-abda-e77be9c8660b admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:06.405111 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1c1e68-b92d-40ce-abda-e77be9c8660b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:06.408274 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1c1e68-b92d-40ce-abda-e77be9c8660b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:06.417290 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d0f981-e0c8-4719-a2f8-12e84009bc98 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:06.424851 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ea718-70da-40ab-96bc-e42033da302b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:06.426034 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ea718-70da-40ab-96bc-e42033da302b admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:06.426520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ea718-70da-40ab-96bc-e42033da302b admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:06.427000 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ea718-70da-40ab-96bc-e42033da302b admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:06.427956 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7ea718-70da-40ab-96bc-e42033da302b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:06.429867 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7ea718-70da-40ab-96bc-e42033da302b admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:06.430157 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 786/1593] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:06.431696 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1c1e68-b92d-40ce-abda-e77be9c8660b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:06.453548 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 808/1594] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:06.453767 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1c1e68-b92d-40ce-abda-e77be9c8660b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:06.455281 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ea718-70da-40ab-96bc-e42033da302b admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:06.463227 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5a68dc-6bae-4fb0-8033-3dba1e4cb988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:06.464449 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a68dc-6bae-4fb0-8033-3dba1e4cb988 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:06.464950 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a68dc-6bae-4fb0-8033-3dba1e4cb988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:06.465446 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a68dc-6bae-4fb0-8033-3dba1e4cb988 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:06.466904 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5a68dc-6bae-4fb0-8033-3dba1e4cb988 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:06.467271 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ea718-70da-40ab-96bc-e42033da302b admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:06.470546 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5a68dc-6bae-4fb0-8033-3dba1e4cb988 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:06.475651 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec747f2-5afc-47fe-8fc5-319aa7fb895c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:06.476504 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec747f2-5afc-47fe-8fc5-319aa7fb895c admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:06.476910 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec747f2-5afc-47fe-8fc5-319aa7fb895c admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:06.477694 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec747f2-5afc-47fe-8fc5-319aa7fb895c admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:06.478590 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec747f2-5afc-47fe-8fc5-319aa7fb895c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:06.480393 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec747f2-5afc-47fe-8fc5-319aa7fb895c admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:06.489864 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:06.491890 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 787/1595] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:06.493472 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5a68dc-6bae-4fb0-8033-3dba1e4cb988 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:06.500176 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 809/1596] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:06.500176 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec747f2-5afc-47fe-8fc5-319aa7fb895c admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:06.512850 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec747f2-5afc-47fe-8fc5-319aa7fb895c admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:06.513198 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5a68dc-6bae-4fb0-8033-3dba1e4cb988 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:06.519662 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b20ad2a-91cd-4c0f-863a-ba8543245b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:06.520647 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b20ad2a-91cd-4c0f-863a-ba8543245b2e admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:06.521353 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b20ad2a-91cd-4c0f-863a-ba8543245b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:06.521558 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c323d-844f-4a57-97bb-611b6d07d55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:06.521950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b20ad2a-91cd-4c0f-863a-ba8543245b2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:06.522729 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b20ad2a-91cd-4c0f-863a-ba8543245b2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:06.522874 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c323d-844f-4a57-97bb-611b6d07d55a admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:06.523294 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c323d-844f-4a57-97bb-611b6d07d55a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:06.523778 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c323d-844f-4a57-97bb-611b6d07d55a admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:06.524782 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b20ad2a-91cd-4c0f-863a-ba8543245b2e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:06.525115 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c323d-844f-4a57-97bb-611b6d07d55a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:06.528323 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c323d-844f-4a57-97bb-611b6d07d55a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:06.542393 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 810/1597] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:06.542393 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b20ad2a-91cd-4c0f-863a-ba8543245b2e admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:06.557145 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 788/1598] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:06.557371 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b20ad2a-91cd-4c0f-863a-ba8543245b2e admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:06.557495 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c323d-844f-4a57-97bb-611b6d07d55a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:06.562360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e927602c-dfbd-4c76-b951-aa183aeda693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:06.563435 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e927602c-dfbd-4c76-b951-aa183aeda693 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:06.563787 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e927602c-dfbd-4c76-b951-aa183aeda693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:06.564323 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e927602c-dfbd-4c76-b951-aa183aeda693 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:06.565155 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e927602c-dfbd-4c76-b951-aa183aeda693 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:06.567123 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e927602c-dfbd-4c76-b951-aa183aeda693 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:06.576721 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c323d-844f-4a57-97bb-611b6d07d55a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:06.584145 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 811/1599] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:06.584145 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e927602c-dfbd-4c76-b951-aa183aeda693 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:06.590135 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed2f36c-4e97-48f9-ae81-6939a8dfcb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:06.591705 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed2f36c-4e97-48f9-ae81-6939a8dfcb4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:06.592214 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed2f36c-4e97-48f9-ae81-6939a8dfcb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:06.592932 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed2f36c-4e97-48f9-ae81-6939a8dfcb4f admin admin] PATH_INFO: `/v3/projects/0c018c1fa6574775b37e28ef9f896390/users/81806b3d747d411f80d0b48c1863d79f/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:06.595637 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e927602c-dfbd-4c76-b951-aa183aeda693 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:06.600695 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb78a8-8929-4e04-9a25-86aa87499274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:06.601566 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb78a8-8929-4e04-9a25-86aa87499274 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:06.601983 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb78a8-8929-4e04-9a25-86aa87499274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:06.602355 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb78a8-8929-4e04-9a25-86aa87499274 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:06.603591 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb78a8-8929-4e04-9a25-86aa87499274 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:06.605370 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb78a8-8929-4e04-9a25-86aa87499274 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:06.619290 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 812/1600] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:06.620506 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb78a8-8929-4e04-9a25-86aa87499274 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:06.623345 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed2f36c-4e97-48f9-ae81-6939a8dfcb4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c018c1fa6574775b37e28ef9f896390, user_id=81806b3d747d411f80d0b48c1863d79f, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:06.629936 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed2f36c-4e97-48f9-ae81-6939a8dfcb4f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:06.634156 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb78a8-8929-4e04-9a25-86aa87499274 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:06.640814 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4352ef-35c6-4460-870c-810dcd752970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:06.641937 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4352ef-35c6-4460-870c-810dcd752970 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:06.642475 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4352ef-35c6-4460-870c-810dcd752970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:06.642867 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4352ef-35c6-4460-870c-810dcd752970 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:06.644075 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4352ef-35c6-4460-870c-810dcd752970 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:06.646289 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4352ef-35c6-4460-870c-810dcd752970 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:06.658047 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 789/1601] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:29:06 2022] PUT /identity/v3/projects/0c018c1fa6574775b37e28ef9f896390/users/81806b3d747d411f80d0b48c1863d79f/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:06.659430 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed2f36c-4e97-48f9-ae81-6939a8dfcb4f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:06.673573 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 813/1602] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:06.673573 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4352ef-35c6-4460-870c-810dcd752970 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:06.682733 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4352ef-35c6-4460-870c-810dcd752970 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:06.756523 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f25b7f-51c3-4a6c-b845-a06f91b249a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:06.757493 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f25b7f-51c3-4a6c-b845-a06f91b249a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:06.757863 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f25b7f-51c3-4a6c-b845-a06f91b249a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:06.758204 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f25b7f-51c3-4a6c-b845-a06f91b249a6 admin admin] PATH_INFO: `/v3/projects/0940f98b5fc841d7a1b32a2a5312fd61/users/6d5ca05069584b089e2d1f26eb3eb003/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:06.765279 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:06.778916 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f25b7f-51c3-4a6c-b845-a06f91b249a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0940f98b5fc841d7a1b32a2a5312fd61, user_id=6d5ca05069584b089e2d1f26eb3eb003, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:06.780238 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: WARNING py.warnings [None req-78f25b7f-51c3-4a6c-b845-a06f91b249a6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 18:29:06.780238 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: warnings.warn(msg) Jun 21 18:29:06.780238 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: Jun 21 18:29:06.782990 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f25b7f-51c3-4a6c-b845-a06f91b249a6 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:06.793933 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed2f36c-4e97-48f9-ae81-6939a8dfcb4f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:06.799355 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7ee12d-346e-4989-a524-aa492441c064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:06.800312 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ee12d-346e-4989-a524-aa492441c064 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:06.800602 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ee12d-346e-4989-a524-aa492441c064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:06.800883 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ee12d-346e-4989-a524-aa492441c064 admin admin] PATH_INFO: `/v3/projects/53e2bb9f793041c2aef6e222b76c3961/users/144c9c31ed9b469aa200dcb16c89ab4e/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:06.812608 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 814/1603] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:29:06 2022] PUT /identity/v3/projects/0940f98b5fc841d7a1b32a2a5312fd61/users/6d5ca05069584b089e2d1f26eb3eb003/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:06.812608 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f25b7f-51c3-4a6c-b845-a06f91b249a6 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:06.831119 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7ee12d-346e-4989-a524-aa492441c064 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53e2bb9f793041c2aef6e222b76c3961, user_id=144c9c31ed9b469aa200dcb16c89ab4e, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:06.836341 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7ee12d-346e-4989-a524-aa492441c064 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:06.868786 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 790/1604] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:29:06 2022] PUT /identity/v3/projects/53e2bb9f793041c2aef6e222b76c3961/users/144c9c31ed9b469aa200dcb16c89ab4e/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:06.869742 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7ee12d-346e-4989-a524-aa492441c064 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:06.946889 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f25b7f-51c3-4a6c-b845-a06f91b249a6 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:07.009999 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:07.011619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7ee12d-346e-4989-a524-aa492441c064 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:07.022862 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f3241e-2511-42f5-9f05-8138e3dbcb9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:07.024295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3241e-2511-42f5-9f05-8138e3dbcb9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:07.025127 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3241e-2511-42f5-9f05-8138e3dbcb9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:07.025546 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3241e-2511-42f5-9f05-8138e3dbcb9a admin admin] PATH_INFO: `/v3/projects/3ec54ca89c0d4dbd822df30b3c0099fe/users/0b750ad5e96c42bc8a5be6b3e3dd6ab5/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:07.029048 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e515ca-828c-44d8-af91-dbf2cb90e82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:07.029577 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e515ca-828c-44d8-af91-dbf2cb90e82a admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:07.029901 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e515ca-828c-44d8-af91-dbf2cb90e82a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:07.030257 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e515ca-828c-44d8-af91-dbf2cb90e82a admin admin] PATH_INFO: `/v3/projects/3c88935a83d24e408c99cb711ae46257/users/acf9a33e9fb941939010989a89a51d38/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:07.046090 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f3241e-2511-42f5-9f05-8138e3dbcb9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ec54ca89c0d4dbd822df30b3c0099fe, user_id=0b750ad5e96c42bc8a5be6b3e3dd6ab5, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:07.048650 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e515ca-828c-44d8-af91-dbf2cb90e82a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c88935a83d24e408c99cb711ae46257, user_id=acf9a33e9fb941939010989a89a51d38, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:07.049414 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f3241e-2511-42f5-9f05-8138e3dbcb9a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:07.052088 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e515ca-828c-44d8-af91-dbf2cb90e82a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:07.080533 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 815/1605] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:29:06 2022] PUT /identity/v3/projects/3c88935a83d24e408c99cb711ae46257/users/acf9a33e9fb941939010989a89a51d38/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:07.080533 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c474c0d-64bd-40bf-98ff-fd56997ae073 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:07.080533 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c474c0d-64bd-40bf-98ff-fd56997ae073 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:07.080533 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c474c0d-64bd-40bf-98ff-fd56997ae073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:07.082346 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 791/1606] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:29:06 2022] PUT /identity/v3/projects/3ec54ca89c0d4dbd822df30b3c0099fe/users/0b750ad5e96c42bc8a5be6b3e3dd6ab5/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:07.082346 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f3241e-2511-42f5-9f05-8138e3dbcb9a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:07.127521 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-9c474c0d-64bd-40bf-98ff-fd56997ae073 None None] MFA Rules not processed for user `81806b3d747d411f80d0b48c1863d79f`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:07.129900 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-9c474c0d-64bd-40bf-98ff-fd56997ae073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:07.247416 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f3241e-2511-42f5-9f05-8138e3dbcb9a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:07.266414 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c7637c-1039-416e-8ce3-7a1376f82278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:07.268487 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c7637c-1039-416e-8ce3-7a1376f82278 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:07.269056 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c7637c-1039-416e-8ce3-7a1376f82278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:07.269573 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c7637c-1039-416e-8ce3-7a1376f82278 admin admin] PATH_INFO: `/v3/projects/a7814d8e77554771b0aad9c0768d1b91/users/11342f921cf846c19bcf64211507b769/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:07.309469 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c7637c-1039-416e-8ce3-7a1376f82278 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7814d8e77554771b0aad9c0768d1b91, user_id=11342f921cf846c19bcf64211507b769, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:07.314959 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c7637c-1039-416e-8ce3-7a1376f82278 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:07.338582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 792/1607] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:29:07 2022] PUT /identity/v3/projects/a7814d8e77554771b0aad9c0768d1b91/users/11342f921cf846c19bcf64211507b769/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:07.340690 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc3db4-5a87-416a-9d80-6bdd02a370d9 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:07.341181 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc3db4-5a87-416a-9d80-6bdd02a370d9 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:07.341802 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc3db4-5a87-416a-9d80-6bdd02a370d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:07.371572 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 816/1608] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:07 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:07.373783 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbdab55-d1d8-4e75-9125-e5dfa6d6b76c None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:07.374831 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbdab55-d1d8-4e75-9125-e5dfa6d6b76c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:07.375145 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbdab55-d1d8-4e75-9125-e5dfa6d6b76c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:07.378550 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-c6cc3db4-5a87-416a-9d80-6bdd02a370d9 None None] MFA Rules not processed for user `6d5ca05069584b089e2d1f26eb3eb003`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:07.380112 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c6cc3db4-5a87-416a-9d80-6bdd02a370d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:07.405238 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-ddbdab55-d1d8-4e75-9125-e5dfa6d6b76c None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:07.406729 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ddbdab55-d1d8-4e75-9125-e5dfa6d6b76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:07.412263 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:07.485497 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 793/1609] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:07 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:07.487677 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ed5a9-4f4a-427d-88e0-41489c2e1012 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:07.488185 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ed5a9-4f4a-427d-88e0-41489c2e1012 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:07.488623 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ed5a9-4f4a-427d-88e0-41489c2e1012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:07.513734 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a94ed5a9-4f4a-427d-88e0-41489c2e1012 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:07.515646 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a94ed5a9-4f4a-427d-88e0-41489c2e1012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:07.524808 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 817/1610] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:07.530890 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec2fa5-f38c-4139-9304-d1d253299066 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:07.531469 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec2fa5-f38c-4139-9304-d1d253299066 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:07.531909 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec2fa5-f38c-4139-9304-d1d253299066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:07.534033 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 794/1611] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:07.549909 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebb1d86-5404-4fa8-b96c-a3456071f127 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:07.550412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebb1d86-5404-4fa8-b96c-a3456071f127 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:07.550682 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebb1d86-5404-4fa8-b96c-a3456071f127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:07.584987 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-9ebb1d86-5404-4fa8-b96c-a3456071f127 None None] MFA Rules not processed for user `11342f921cf846c19bcf64211507b769`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:07.586557 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9ebb1d86-5404-4fa8-b96c-a3456071f127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:07.590851 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-21ec2fa5-f38c-4139-9304-d1d253299066 None None] MFA Rules not processed for user `0b750ad5e96c42bc8a5be6b3e3dd6ab5`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:07.592680 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-21ec2fa5-f38c-4139-9304-d1d253299066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:07.669441 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 795/1612] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:07.675362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebb1d86-5404-4fa8-b96c-a3456071f127 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:07.696129 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:07.705060 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebb1d86-5404-4fa8-b96c-a3456071f127 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:07.713220 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a9ed4-80fa-4ee5-aa45-8aeae233f145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:07.714442 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a9ed4-80fa-4ee5-aa45-8aeae233f145 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:07.714893 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a9ed4-80fa-4ee5-aa45-8aeae233f145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:07.715317 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a9ed4-80fa-4ee5-aa45-8aeae233f145 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:07.716457 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13a9ed4-80fa-4ee5-aa45-8aeae233f145 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:07.719113 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13a9ed4-80fa-4ee5-aa45-8aeae233f145 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:07.726473 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 818/1613] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:07 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:07.729232 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ec2fa5-f38c-4139-9304-d1d253299066 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:07.745534 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ec2fa5-f38c-4139-9304-d1d253299066 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:07.753482 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b71e31-2c4c-4ec9-8f34-007e9cdc520d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:07.754277 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b71e31-2c4c-4ec9-8f34-007e9cdc520d admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:07.754590 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b71e31-2c4c-4ec9-8f34-007e9cdc520d admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:07.754899 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b71e31-2c4c-4ec9-8f34-007e9cdc520d admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:07.755661 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b71e31-2c4c-4ec9-8f34-007e9cdc520d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:07.759716 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b71e31-2c4c-4ec9-8f34-007e9cdc520d admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:07.788084 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 796/1614] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:29:07 2022] POST /identity/v3/projects => generated 401 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:07.789872 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a9ed4-80fa-4ee5-aa45-8aeae233f145 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:07.815915 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 819/1615] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:29:07 2022] POST /identity/v3/projects => generated 363 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:07.816860 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b71e31-2c4c-4ec9-8f34-007e9cdc520d admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:07.916263 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b71e31-2c4c-4ec9-8f34-007e9cdc520d admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:07.922382 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a9ed4-80fa-4ee5-aa45-8aeae233f145 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:07.927229 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d70c06c-683d-47e8-99e3-06031911de9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:07.929544 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70c06c-683d-47e8-99e3-06031911de9d service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:07.929544 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70c06c-683d-47e8-99e3-06031911de9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:07.929544 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70c06c-683d-47e8-99e3-06031911de9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:07.935014 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b796deb6-2bc7-4b48-a5b5-67d62ca3af45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:07.936513 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b796deb6-2bc7-4b48-a5b5-67d62ca3af45 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:07.936513 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b796deb6-2bc7-4b48-a5b5-67d62ca3af45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:07.936513 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b796deb6-2bc7-4b48-a5b5-67d62ca3af45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:08.006199 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:08.040821 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b796deb6-2bc7-4b48-a5b5-67d62ca3af45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:08.043449 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b796deb6-2bc7-4b48-a5b5-67d62ca3af45 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:08.065145 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 797/1616] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:29:08.066302 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b796deb6-2bc7-4b48-a5b5-67d62ca3af45 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:08.167789 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b796deb6-2bc7-4b48-a5b5-67d62ca3af45 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:08.173401 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd27f097-aeb8-4496-a23f-a686c9461414 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:08.176198 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27f097-aeb8-4496-a23f-a686c9461414 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:08.176995 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27f097-aeb8-4496-a23f-a686c9461414 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:08.177404 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27f097-aeb8-4496-a23f-a686c9461414 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:08.188172 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d70c06c-683d-47e8-99e3-06031911de9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:08.190712 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d70c06c-683d-47e8-99e3-06031911de9d service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:08.205863 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 820/1617] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:29:08.208267 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd39dc2-be9e-4757-bc02-864b91e36575 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:08.208599 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd39dc2-be9e-4757-bc02-864b91e36575 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:08.208985 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd39dc2-be9e-4757-bc02-864b91e36575 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:08.217419 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 821/1618] 10.222.0.87 () {54 vars in 961 bytes} [Tue Jun 21 18:29:08 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:29:08.219254 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd39dc2-be9e-4757-bc02-864b91e36575 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:08.239244 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd39dc2-be9e-4757-bc02-864b91e36575 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:08.248379 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fcd5eb-70c1-4fcd-b297-9ea1d24e1aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:08.249090 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fcd5eb-70c1-4fcd-b297-9ea1d24e1aa5 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:08.249475 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fcd5eb-70c1-4fcd-b297-9ea1d24e1aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:08.249985 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fcd5eb-70c1-4fcd-b297-9ea1d24e1aa5 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:08.250991 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fcd5eb-70c1-4fcd-b297-9ea1d24e1aa5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:08.254075 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fcd5eb-70c1-4fcd-b297-9ea1d24e1aa5 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:08.265672 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd27f097-aeb8-4496-a23f-a686c9461414 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:08.269937 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd27f097-aeb8-4496-a23f-a686c9461414 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:08.282173 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 798/1619] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:08 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:29:08.282734 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd27f097-aeb8-4496-a23f-a686c9461414 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:08.286059 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 822/1620] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:29:08 2022] POST /identity/v3/users => generated 307 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:08.289737 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac90a4b-543e-46fa-823b-98f8f31478d2 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:08.290609 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac90a4b-543e-46fa-823b-98f8f31478d2 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:08.291138 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac90a4b-543e-46fa-823b-98f8f31478d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:08.298743 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd27f097-aeb8-4496-a23f-a686c9461414 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:08.305143 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced2b9e7-4018-4db5-8225-f0522a9a47d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:08.305888 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2b9e7-4018-4db5-8225-f0522a9a47d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:08.306176 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2b9e7-4018-4db5-8225-f0522a9a47d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:08.306451 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2b9e7-4018-4db5-8225-f0522a9a47d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:08.307517 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced2b9e7-4018-4db5-8225-f0522a9a47d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:08.309771 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced2b9e7-4018-4db5-8225-f0522a9a47d6 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:08.321242 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-eac90a4b-543e-46fa-823b-98f8f31478d2 None None] MFA Rules not processed for user `a1b3fa67905c4027b3656ac36e2b76c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:08.323772 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-eac90a4b-543e-46fa-823b-98f8f31478d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:08.340669 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 799/1621] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:29:08 2022] POST /identity/v3/users => generated 288 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:08.344152 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced2b9e7-4018-4db5-8225-f0522a9a47d6 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:08.374758 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced2b9e7-4018-4db5-8225-f0522a9a47d6 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:08.389102 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2008e8ff-2f7d-45f2-bd9b-46bf5ca1292d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:08.391157 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008e8ff-2f7d-45f2-bd9b-46bf5ca1292d admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:08.392947 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008e8ff-2f7d-45f2-bd9b-46bf5ca1292d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:08.393435 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008e8ff-2f7d-45f2-bd9b-46bf5ca1292d admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:08.396465 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2008e8ff-2f7d-45f2-bd9b-46bf5ca1292d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:08.400046 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2008e8ff-2f7d-45f2-bd9b-46bf5ca1292d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:08.432499 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 800/1622] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:08 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:08.435705 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2008e8ff-2f7d-45f2-bd9b-46bf5ca1292d admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:08.477528 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2008e8ff-2f7d-45f2-bd9b-46bf5ca1292d admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:08.507860 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d56844-5fd8-4152-916c-6a079cfcfe0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:08.509034 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d56844-5fd8-4152-916c-6a079cfcfe0c admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:08.509521 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d56844-5fd8-4152-916c-6a079cfcfe0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:08.510038 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d56844-5fd8-4152-916c-6a079cfcfe0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:08.511436 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d56844-5fd8-4152-916c-6a079cfcfe0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:08.514606 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:08.531419 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d56844-5fd8-4152-916c-6a079cfcfe0c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:08.601366 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 801/1623] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:08 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:08.602391 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d56844-5fd8-4152-916c-6a079cfcfe0c admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:08.614377 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 823/1624] 10.222.0.87 () {58 vars in 1058 bytes} [Tue Jun 21 18:29:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:08.615957 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac90a4b-543e-46fa-823b-98f8f31478d2 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:08.646623 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d56844-5fd8-4152-916c-6a079cfcfe0c admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:08.657302 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac90a4b-543e-46fa-823b-98f8f31478d2 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:08.665808 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa6641a-3880-4446-b775-a6305857453c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:08.667448 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6641a-3880-4446-b775-a6305857453c admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:08.668141 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6641a-3880-4446-b775-a6305857453c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:08.668650 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6641a-3880-4446-b775-a6305857453c admin admin] PATH_INFO: `/v3/projects/1084d25b84f84802b848e5390f7eb965/users/3f5c560e676f4fe7a59827a5df24688c/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:08.676363 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d44387-4697-4637-acb4-e57c5b93d267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:08.678090 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d44387-4697-4637-acb4-e57c5b93d267 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:08.678580 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d44387-4697-4637-acb4-e57c5b93d267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:08.679060 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d44387-4697-4637-acb4-e57c5b93d267 admin admin] PATH_INFO: `/v3/projects/f7881e9292d046718a487d6021876316/users/c4767cb6107c4a70becb08035342e66d/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:08.701290 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa6641a-3880-4446-b775-a6305857453c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1084d25b84f84802b848e5390f7eb965, user_id=3f5c560e676f4fe7a59827a5df24688c, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:08.703305 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d44387-4697-4637-acb4-e57c5b93d267 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7881e9292d046718a487d6021876316, user_id=c4767cb6107c4a70becb08035342e66d, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:08.706306 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d44387-4697-4637-acb4-e57c5b93d267 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:08.708153 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa6641a-3880-4446-b775-a6305857453c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:08.730770 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 824/1625] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:29:08 2022] PUT /identity/v3/projects/f7881e9292d046718a487d6021876316/users/c4767cb6107c4a70becb08035342e66d/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:08.730770 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d44387-4697-4637-acb4-e57c5b93d267 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:08.755081 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 802/1626] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:29:08 2022] PUT /identity/v3/projects/1084d25b84f84802b848e5390f7eb965/users/3f5c560e676f4fe7a59827a5df24688c/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:08.756639 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa6641a-3880-4446-b775-a6305857453c admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:08.851222 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d44387-4697-4637-acb4-e57c5b93d267 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:08.890836 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:08.986641 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1131f9d0-26ef-462d-949d-c47a3f3c0f56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:08.987889 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1131f9d0-26ef-462d-949d-c47a3f3c0f56 service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:08.988582 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1131f9d0-26ef-462d-949d-c47a3f3c0f56 service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:08.989547 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1131f9d0-26ef-462d-949d-c47a3f3c0f56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:09.026483 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa6641a-3880-4446-b775-a6305857453c admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:09.043315 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1282c-c619-42f9-9730-59face4c089e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:09.044489 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1282c-c619-42f9-9730-59face4c089e admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:09.046074 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1282c-c619-42f9-9730-59face4c089e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:09.046619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1282c-c619-42f9-9730-59face4c089e admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:09.048845 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea1282c-c619-42f9-9730-59face4c089e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:09.052503 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea1282c-c619-42f9-9730-59face4c089e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:09.082736 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 803/1627] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:08 2022] GET /identity/v3/roles => generated 1420 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:09.085876 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1282c-c619-42f9-9730-59face4c089e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:09.109920 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1282c-c619-42f9-9730-59face4c089e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:09.122592 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d0a537-ab96-48ce-aa32-1fbe0b63a47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:09.124573 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0a537-ab96-48ce-aa32-1fbe0b63a47a admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:09.125342 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0a537-ab96-48ce-aa32-1fbe0b63a47a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:09.126046 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0a537-ab96-48ce-aa32-1fbe0b63a47a admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:09.128772 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d0a537-ab96-48ce-aa32-1fbe0b63a47a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:09.133425 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d0a537-ab96-48ce-aa32-1fbe0b63a47a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:09.169306 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 804/1628] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:09 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:09.169306 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d0a537-ab96-48ce-aa32-1fbe0b63a47a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:09.183341 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1131f9d0-26ef-462d-949d-c47a3f3c0f56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:09.188044 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1131f9d0-26ef-462d-949d-c47a3f3c0f56 service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:09.199289 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d0a537-ab96-48ce-aa32-1fbe0b63a47a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:09.212635 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6868cd-bd48-4a85-a054-8bdfabf957b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:09.213034 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 825/1629] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:29:09.214163 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6868cd-bd48-4a85-a054-8bdfabf957b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:09.214620 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1131f9d0-26ef-462d-949d-c47a3f3c0f56 service nova] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:09.214765 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6868cd-bd48-4a85-a054-8bdfabf957b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:09.215203 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6868cd-bd48-4a85-a054-8bdfabf957b2 admin admin] PATH_INFO: `/v3/projects/f7881e9292d046718a487d6021876316/users/c4767cb6107c4a70becb08035342e66d/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:09.221017 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6868cd-bd48-4a85-a054-8bdfabf957b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7881e9292d046718a487d6021876316, user_id=c4767cb6107c4a70becb08035342e66d, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:09.225107 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:09.234547 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6868cd-bd48-4a85-a054-8bdfabf957b2 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:09.243803 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1131f9d0-26ef-462d-949d-c47a3f3c0f56 service nova] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:09.502792 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6673dd-8ce8-4687-b56c-9c86ec27de85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:09.508453 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6673dd-8ce8-4687-b56c-9c86ec27de85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:09.512433 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6673dd-8ce8-4687-b56c-9c86ec27de85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:09.513104 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6673dd-8ce8-4687-b56c-9c86ec27de85 admin admin] PATH_INFO: `/v3/projects/1084d25b84f84802b848e5390f7eb965/users/3f5c560e676f4fe7a59827a5df24688c/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:09.525559 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6673dd-8ce8-4687-b56c-9c86ec27de85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1084d25b84f84802b848e5390f7eb965, user_id=3f5c560e676f4fe7a59827a5df24688c, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:09.543416 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6673dd-8ce8-4687-b56c-9c86ec27de85 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:09.582701 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 805/1630] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:29:09 2022] PUT /identity/v3/projects/f7881e9292d046718a487d6021876316/users/c4767cb6107c4a70becb08035342e66d/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:09.599429 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 826/1631] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:29:09 2022] PUT /identity/v3/projects/1084d25b84f84802b848e5390f7eb965/users/3f5c560e676f4fe7a59827a5df24688c/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:09.607927 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e6717-e91d-4128-bd43-28ec77675321 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:09.608529 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e6717-e91d-4128-bd43-28ec77675321 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:09.609564 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e6717-e91d-4128-bd43-28ec77675321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:09.621086 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5f140-5ad2-45d4-baca-7641a467ae95 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:09.622032 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5f140-5ad2-45d4-baca-7641a467ae95 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:09.622902 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5f140-5ad2-45d4-baca-7641a467ae95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:09.718873 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d75e6717-e91d-4128-bd43-28ec77675321 None None] MFA Rules not processed for user `c4767cb6107c4a70becb08035342e66d`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:09.723245 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d75e6717-e91d-4128-bd43-28ec77675321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:09.752885 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-2ca5f140-5ad2-45d4-baca-7641a467ae95 None None] MFA Rules not processed for user `3f5c560e676f4fe7a59827a5df24688c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:09.755789 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-2ca5f140-5ad2-45d4-baca-7641a467ae95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:10.040082 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 806/1632] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:09 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 434 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:10.065516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 827/1633] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:09 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 450 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:10.073252 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bcb9dc-d62d-4da6-aa08-166f4a53280b None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:10.073787 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bcb9dc-d62d-4da6-aa08-166f4a53280b None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:10.074446 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e6717-e91d-4128-bd43-28ec77675321 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:10.074651 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bcb9dc-d62d-4da6-aa08-166f4a53280b None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:10.078482 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 828/1634] 10.222.0.87 () {54 vars in 912 bytes} [Tue Jun 21 18:29:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:29:10.088348 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e391f97d-20b6-4058-9398-ae3fd91a457b None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:10.088848 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e391f97d-20b6-4058-9398-ae3fd91a457b None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:10.089386 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e391f97d-20b6-4058-9398-ae3fd91a457b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:10.159407 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-e391f97d-20b6-4058-9398-ae3fd91a457b None None] MFA Rules not processed for user `a1b3fa67905c4027b3656ac36e2b76c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:10.161918 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e391f97d-20b6-4058-9398-ae3fd91a457b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:10.406778 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e6717-e91d-4128-bd43-28ec77675321 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:10.444125 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c55287-b77a-4136-964e-2d6c7c9c978e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:10.444748 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c55287-b77a-4136-964e-2d6c7c9c978e service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:10.445040 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c55287-b77a-4136-964e-2d6c7c9c978e service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:10.454336 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c55287-b77a-4136-964e-2d6c7c9c978e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:10.522943 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c55287-b77a-4136-964e-2d6c7c9c978e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:10.525705 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c55287-b77a-4136-964e-2d6c7c9c978e service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:10.570126 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 829/1635] 10.222.0.87 () {58 vars in 1009 bytes} [Tue Jun 21 18:29:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 484 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:10.576192 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e391f97d-20b6-4058-9398-ae3fd91a457b None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:10.592754 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 807/1636] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:10 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:29:10.617949 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e391f97d-20b6-4058-9398-ae3fd91a457b None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:10.633289 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c00fd22-0f3d-4892-8921-0e23e406a0b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:10.634471 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00fd22-0f3d-4892-8921-0e23e406a0b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:10.634881 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00fd22-0f3d-4892-8921-0e23e406a0b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:10.635292 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00fd22-0f3d-4892-8921-0e23e406a0b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:10.682332 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c00fd22-0f3d-4892-8921-0e23e406a0b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:10.684572 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c00fd22-0f3d-4892-8921-0e23e406a0b7 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:10.704071 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:10.712826 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c55287-b77a-4136-964e-2d6c7c9c978e service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:10.727326 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 830/1637] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:10 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:29:10.736152 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c55287-b77a-4136-964e-2d6c7c9c978e service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:10.755579 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8786de-576d-42e5-bef9-eaabdc68210e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:10.756739 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8786de-576d-42e5-bef9-eaabdc68210e service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:10.757238 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8786de-576d-42e5-bef9-eaabdc68210e service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:10.757730 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8786de-576d-42e5-bef9-eaabdc68210e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:10.801976 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8786de-576d-42e5-bef9-eaabdc68210e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:10.805037 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8786de-576d-42e5-bef9-eaabdc68210e service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:10.935635 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 808/1638] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:29:11.214540 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c00fd22-0f3d-4892-8921-0e23e406a0b7 service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:11.238980 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c00fd22-0f3d-4892-8921-0e23e406a0b7 service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:11.247237 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86c1714-2784-4bc0-ac1b-dab94bd2e1bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:11.249228 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c1714-2784-4bc0-ac1b-dab94bd2e1bb service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:11.249228 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c1714-2784-4bc0-ac1b-dab94bd2e1bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:11.249228 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c1714-2784-4bc0-ac1b-dab94bd2e1bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:11.350792 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86c1714-2784-4bc0-ac1b-dab94bd2e1bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:11.352376 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86c1714-2784-4bc0-ac1b-dab94bd2e1bb service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:11.369434 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 831/1639] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:11 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:29:12.024174 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8786de-576d-42e5-bef9-eaabdc68210e service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:12.081529 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:12.252950 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8786de-576d-42e5-bef9-eaabdc68210e service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:12.272577 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21889eb4-eb2b-4d60-9ccf-cb424b0455b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:12.274722 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21889eb4-eb2b-4d60-9ccf-cb424b0455b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:12.275195 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21889eb4-eb2b-4d60-9ccf-cb424b0455b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:12.275638 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21889eb4-eb2b-4d60-9ccf-cb424b0455b5 admin admin] PATH_INFO: `/v3/users/81806b3d747d411f80d0b48c1863d79f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:12.290492 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21889eb4-eb2b-4d60-9ccf-cb424b0455b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81806b3d747d411f80d0b48c1863d79f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:12.294845 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21889eb4-eb2b-4d60-9ccf-cb424b0455b5 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:12.458926 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-21889eb4-eb2b-4d60-9ccf-cb424b0455b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81806b3d747d411f80d0b48c1863d79f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:12.499586 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-21889eb4-eb2b-4d60-9ccf-cb424b0455b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81806b3d747d411f80d0b48c1863d79f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:12.500644 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-21889eb4-eb2b-4d60-9ccf-cb424b0455b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81806b3d747d411f80d0b48c1863d79f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:12.501736 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-21889eb4-eb2b-4d60-9ccf-cb424b0455b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81806b3d747d411f80d0b48c1863d79f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:12.528254 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-21889eb4-eb2b-4d60-9ccf-cb424b0455b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81806b3d747d411f80d0b48c1863d79f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:12.561252 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 809/1640] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:29:12 2022] DELETE /identity/v3/users/81806b3d747d411f80d0b48c1863d79f => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:12.565570 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99577b4-9778-4e6e-ad94-17c10b414b5b None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:12.567310 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99577b4-9778-4e6e-ad94-17c10b414b5b None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:12.568009 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99577b4-9778-4e6e-ad94-17c10b414b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:12.618394 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-a99577b4-9778-4e6e-ad94-17c10b414b5b None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:12.620795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a99577b4-9778-4e6e-ad94-17c10b414b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:12.694878 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895f5b3-a6c0-455f-b68d-f5780ade592f None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:12.695297 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895f5b3-a6c0-455f-b68d-f5780ade592f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:12.695632 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895f5b3-a6c0-455f-b68d-f5780ade592f None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:12.698176 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 810/1641] 10.222.0.87 () {54 vars in 961 bytes} [Tue Jun 21 18:29:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:29:12.726326 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97274f68-8180-415c-aaf9-13f83d555a04 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:12.726785 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97274f68-8180-415c-aaf9-13f83d555a04 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:12.727081 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97274f68-8180-415c-aaf9-13f83d555a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:12.795675 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-97274f68-8180-415c-aaf9-13f83d555a04 None None] MFA Rules not processed for user `a1b3fa67905c4027b3656ac36e2b76c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:12.801378 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-97274f68-8180-415c-aaf9-13f83d555a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:12.815459 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 832/1642] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:12.884814 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99577b4-9778-4e6e-ad94-17c10b414b5b None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:12.887999 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a99577b4-9778-4e6e-ad94-17c10b414b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:13.049102 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99577b4-9778-4e6e-ad94-17c10b414b5b None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:13.059528 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb182ae-aa77-4489-a66c-2f410233cd45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:13.060811 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb182ae-aa77-4489-a66c-2f410233cd45 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:13.061611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb182ae-aa77-4489-a66c-2f410233cd45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:13.062514 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb182ae-aa77-4489-a66c-2f410233cd45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:13.090003 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb182ae-aa77-4489-a66c-2f410233cd45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:13.093195 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb182ae-aa77-4489-a66c-2f410233cd45 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:13.117330 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 833/1643] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:29:13.165313 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 811/1644] 10.222.0.87 () {58 vars in 1058 bytes} [Tue Jun 21 18:29:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 441 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:29:13.178670 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2118f3b-81d3-446a-bfc8-66c169560949 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:13.178975 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2118f3b-81d3-446a-bfc8-66c169560949 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:13.179513 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2118f3b-81d3-446a-bfc8-66c169560949 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:13.183457 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 834/1645] 10.222.0.87 () {54 vars in 912 bytes} [Tue Jun 21 18:29:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:29:13.199537 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b424f-a213-4e9c-bff2-fdfb5ed2608a None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:13.200099 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b424f-a213-4e9c-bff2-fdfb5ed2608a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:13.200581 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b424f-a213-4e9c-bff2-fdfb5ed2608a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:13.271518 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-690b424f-a213-4e9c-bff2-fdfb5ed2608a None None] MFA Rules not processed for user `a1b3fa67905c4027b3656ac36e2b76c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:13.277046 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-690b424f-a213-4e9c-bff2-fdfb5ed2608a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:13.370427 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 812/1646] 10.222.0.87 () {58 vars in 1009 bytes} [Tue Jun 21 18:29:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:29:13.435368 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2118f3b-81d3-446a-bfc8-66c169560949 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:13.438289 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a2118f3b-81d3-446a-bfc8-66c169560949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:13.611253 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:13.646442 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2118f3b-81d3-446a-bfc8-66c169560949 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:13.664403 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e55e5b-3ae4-4909-8584-12d4faed77bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:13.678422 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e55e5b-3ae4-4909-8584-12d4faed77bc service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:13.679265 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e55e5b-3ae4-4909-8584-12d4faed77bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:13.679991 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e55e5b-3ae4-4909-8584-12d4faed77bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:13.728239 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e55e5b-3ae4-4909-8584-12d4faed77bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:13.741381 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e55e5b-3ae4-4909-8584-12d4faed77bc service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:13.824639 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 835/1647] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:29:15.679387 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b424f-a213-4e9c-bff2-fdfb5ed2608a None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:15.682265 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-690b424f-a213-4e9c-bff2-fdfb5ed2608a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:15.722220 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b424f-a213-4e9c-bff2-fdfb5ed2608a None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:15.746802 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c4ebef-4e2d-4c25-8965-e22a0f8c0e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:15.748405 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c4ebef-4e2d-4c25-8965-e22a0f8c0e90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:15.748925 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c4ebef-4e2d-4c25-8965-e22a0f8c0e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:15.749428 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c4ebef-4e2d-4c25-8965-e22a0f8c0e90 admin admin] PATH_INFO: `/v3/projects/0c018c1fa6574775b37e28ef9f896390` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:15.759253 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c4ebef-4e2d-4c25-8965-e22a0f8c0e90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c018c1fa6574775b37e28ef9f896390)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:15.761968 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: WARNING py.warnings [None req-e9c4ebef-4e2d-4c25-8965-e22a0f8c0e90 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 18:29:15.761968 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: warnings.warn(msg) Jun 21 18:29:15.761968 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: Jun 21 18:29:15.767659 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c4ebef-4e2d-4c25-8965-e22a0f8c0e90 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:15.823942 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:16.071059 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e9c4ebef-4e2d-4c25-8965-e22a0f8c0e90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c018c1fa6574775b37e28ef9f896390', 'request_id': 'req-e9c4ebef-4e2d-4c25-8965-e22a0f8c0e90'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:16.098878 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e9c4ebef-4e2d-4c25-8965-e22a0f8c0e90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c018c1fa6574775b37e28ef9f896390', 'request_id': 'req-e9c4ebef-4e2d-4c25-8965-e22a0f8c0e90'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:16.113446 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e9c4ebef-4e2d-4c25-8965-e22a0f8c0e90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c018c1fa6574775b37e28ef9f896390', 'request_id': 'req-e9c4ebef-4e2d-4c25-8965-e22a0f8c0e90'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:16.155483 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e9c4ebef-4e2d-4c25-8965-e22a0f8c0e90 admin admin] The token cache is being invalidate because project 0c018c1fa6574775b37e28ef9f896390 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:29:16.156097 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e9c4ebef-4e2d-4c25-8965-e22a0f8c0e90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:16.157229 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e9c4ebef-4e2d-4c25-8965-e22a0f8c0e90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:16.165710 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 813/1648] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:29:15 2022] DELETE /identity/v3/projects/0c018c1fa6574775b37e28ef9f896390 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:16.166103 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e55e5b-3ae4-4909-8584-12d4faed77bc service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:16.187212 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e55e5b-3ae4-4909-8584-12d4faed77bc service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:16.451404 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c6db2b-d42a-471d-924b-665769bb9a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:16.453466 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6db2b-d42a-471d-924b-665769bb9a8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:16.454408 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6db2b-d42a-471d-924b-665769bb9a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:16.455168 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6db2b-d42a-471d-924b-665769bb9a8e admin admin] PATH_INFO: `/v3/users/3f5c560e676f4fe7a59827a5df24688c` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:16.466029 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c6db2b-d42a-471d-924b-665769bb9a8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f5c560e676f4fe7a59827a5df24688c)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:16.470136 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: WARNING py.warnings [None req-c0c6db2b-d42a-471d-924b-665769bb9a8e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 18:29:16.470136 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: warnings.warn(msg) Jun 21 18:29:16.470136 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: Jun 21 18:29:16.484902 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c6db2b-d42a-471d-924b-665769bb9a8e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:16.723007 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c0c6db2b-d42a-471d-924b-665769bb9a8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f5c560e676f4fe7a59827a5df24688c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:16.811465 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c0c6db2b-d42a-471d-924b-665769bb9a8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f5c560e676f4fe7a59827a5df24688c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:16.812795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c0c6db2b-d42a-471d-924b-665769bb9a8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f5c560e676f4fe7a59827a5df24688c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:16.814800 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c0c6db2b-d42a-471d-924b-665769bb9a8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f5c560e676f4fe7a59827a5df24688c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:16.838298 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c0c6db2b-d42a-471d-924b-665769bb9a8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f5c560e676f4fe7a59827a5df24688c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:16.902425 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 836/1649] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:29:16 2022] DELETE /identity/v3/users/3f5c560e676f4fe7a59827a5df24688c => generated 0 bytes in 737 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:18.155929 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c4ebef-4e2d-4c25-8965-e22a0f8c0e90 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:18.159073 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e9c4ebef-4e2d-4c25-8965-e22a0f8c0e90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:18.226847 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c4ebef-4e2d-4c25-8965-e22a0f8c0e90 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:18.470900 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b2d1cf-4763-4f2c-9675-c497432f6d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:18.481626 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b2d1cf-4763-4f2c-9675-c497432f6d5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:18.482538 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b2d1cf-4763-4f2c-9675-c497432f6d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:18.482965 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b2d1cf-4763-4f2c-9675-c497432f6d5e admin admin] PATH_INFO: `/v3/projects/1084d25b84f84802b848e5390f7eb965` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:18.486554 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b2d1cf-4763-4f2c-9675-c497432f6d5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1084d25b84f84802b848e5390f7eb965)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:18.488884 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b2d1cf-4763-4f2c-9675-c497432f6d5e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:18.599152 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-96b2d1cf-4763-4f2c-9675-c497432f6d5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1084d25b84f84802b848e5390f7eb965', 'request_id': 'req-96b2d1cf-4763-4f2c-9675-c497432f6d5e'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:18.613928 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-96b2d1cf-4763-4f2c-9675-c497432f6d5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1084d25b84f84802b848e5390f7eb965', 'request_id': 'req-96b2d1cf-4763-4f2c-9675-c497432f6d5e'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:18.635486 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-96b2d1cf-4763-4f2c-9675-c497432f6d5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1084d25b84f84802b848e5390f7eb965', 'request_id': 'req-96b2d1cf-4763-4f2c-9675-c497432f6d5e'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:18.675011 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-96b2d1cf-4763-4f2c-9675-c497432f6d5e admin admin] The token cache is being invalidate because project 1084d25b84f84802b848e5390f7eb965 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:29:18.675531 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-96b2d1cf-4763-4f2c-9675-c497432f6d5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:18.677526 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-96b2d1cf-4763-4f2c-9675-c497432f6d5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:18.687341 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 814/1650] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:29:18 2022] DELETE /identity/v3/projects/1084d25b84f84802b848e5390f7eb965 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:18.689743 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbafc58-6fe1-4b32-aefc-93b970a763cc None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:18.690496 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbafc58-6fe1-4b32-aefc-93b970a763cc None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:18.690626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbafc58-6fe1-4b32-aefc-93b970a763cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:18.734391 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-bbbafc58-6fe1-4b32-aefc-93b970a763cc None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:18.738790 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-bbbafc58-6fe1-4b32-aefc-93b970a763cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:19.010059 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 837/1651] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:19.015781 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd076b40-5bee-4a56-928d-2a5c7e887c3d None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:19.016197 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd076b40-5bee-4a56-928d-2a5c7e887c3d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:19.016690 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd076b40-5bee-4a56-928d-2a5c7e887c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:19.096610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-fd076b40-5bee-4a56-928d-2a5c7e887c3d None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:19.101503 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fd076b40-5bee-4a56-928d-2a5c7e887c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:19.162702 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 815/1652] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:29:19.164631 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbafc58-6fe1-4b32-aefc-93b970a763cc None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:19.173917 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:19.203421 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbafc58-6fe1-4b32-aefc-93b970a763cc None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:19.217201 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11254327-b983-4330-af52-0fd56fca4a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:19.218130 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11254327-b983-4330-af52-0fd56fca4a05 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:19.218711 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11254327-b983-4330-af52-0fd56fca4a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:19.219300 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11254327-b983-4330-af52-0fd56fca4a05 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:19.220577 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11254327-b983-4330-af52-0fd56fca4a05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:29:19.220947 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11254327-b983-4330-af52-0fd56fca4a05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:19.222750 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11254327-b983-4330-af52-0fd56fca4a05 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:19.264757 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 838/1653] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:29:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:29:19.265452 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd076b40-5bee-4a56-928d-2a5c7e887c3d None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:19.303206 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd076b40-5bee-4a56-928d-2a5c7e887c3d None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:19.327971 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fee979-07ef-466c-943f-ace04ab81296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:19.329768 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fee979-07ef-466c-943f-ace04ab81296 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:19.330675 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fee979-07ef-466c-943f-ace04ab81296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:19.332322 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fee979-07ef-466c-943f-ace04ab81296 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:19.334013 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fee979-07ef-466c-943f-ace04ab81296 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:19.338341 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fee979-07ef-466c-943f-ace04ab81296 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:19.426997 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 816/1654] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:29:19 2022] POST /identity/v3/projects => generated 363 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:19.431912 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11254327-b983-4330-af52-0fd56fca4a05 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:19.462179 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11254327-b983-4330-af52-0fd56fca4a05 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:19.584298 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b7c4d-e879-4d96-9cb1-29fa2d2543af None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:19.585214 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b7c4d-e879-4d96-9cb1-29fa2d2543af None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:19.585996 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b7c4d-e879-4d96-9cb1-29fa2d2543af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:19.597238 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:19.642281 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8ce3c6-0757-4ba3-91a6-fa426abc58b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:19.644532 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ce3c6-0757-4ba3-91a6-fa426abc58b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:19.645147 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ce3c6-0757-4ba3-91a6-fa426abc58b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:19.645861 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ce3c6-0757-4ba3-91a6-fa426abc58b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:19.647042 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8ce3c6-0757-4ba3-91a6-fa426abc58b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:19.649939 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8ce3c6-0757-4ba3-91a6-fa426abc58b2 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:19.650385 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-7b9b7c4d-e879-4d96-9cb1-29fa2d2543af None None] MFA Rules not processed for user `de09b91800314d06aeddafe3c12a77b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:19.651955 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7b9b7c4d-e879-4d96-9cb1-29fa2d2543af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:19.686327 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 839/1655] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:29:19 2022] POST /identity/v3/users => generated 282 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:19.691713 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8ce3c6-0757-4ba3-91a6-fa426abc58b2 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:19.716376 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8ce3c6-0757-4ba3-91a6-fa426abc58b2 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:19.725671 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1967e587-b952-4cf3-870e-66b0f25c11df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:19.727082 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967e587-b952-4cf3-870e-66b0f25c11df admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:19.727749 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967e587-b952-4cf3-870e-66b0f25c11df admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:19.728243 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967e587-b952-4cf3-870e-66b0f25c11df admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:19.729592 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1967e587-b952-4cf3-870e-66b0f25c11df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:19.731797 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1967e587-b952-4cf3-870e-66b0f25c11df admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:19.749583 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 840/1656] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:19 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:19.753716 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1967e587-b952-4cf3-870e-66b0f25c11df admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:19.770533 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1967e587-b952-4cf3-870e-66b0f25c11df admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:19.777011 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b4a1c2-800a-4f80-9d4f-c305cfa30907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:19.778878 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b4a1c2-800a-4f80-9d4f-c305cfa30907 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:19.779385 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b4a1c2-800a-4f80-9d4f-c305cfa30907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:19.780097 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b4a1c2-800a-4f80-9d4f-c305cfa30907 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:19.781901 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b4a1c2-800a-4f80-9d4f-c305cfa30907 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:19.784663 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b4a1c2-800a-4f80-9d4f-c305cfa30907 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:19.799720 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 841/1657] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:19 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:19.805258 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b4a1c2-800a-4f80-9d4f-c305cfa30907 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:19.818147 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 817/1658] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:19.826255 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81d2a8-a9a1-4200-8965-95862c4902c9 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:19.826605 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81d2a8-a9a1-4200-8965-95862c4902c9 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:19.826807 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81d2a8-a9a1-4200-8965-95862c4902c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:19.829453 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b4a1c2-800a-4f80-9d4f-c305cfa30907 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:19.843815 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba796b1-a467-4cdb-934d-bc0b66e9c307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:19.844860 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba796b1-a467-4cdb-934d-bc0b66e9c307 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:19.845060 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba796b1-a467-4cdb-934d-bc0b66e9c307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:19.845336 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba796b1-a467-4cdb-934d-bc0b66e9c307 admin admin] PATH_INFO: `/v3/projects/cec0bb6f1f3046b58cf9ae341de94dfd/users/c9644fcbc1634e38acb75492a10ca32c/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:19.863277 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-cd81d2a8-a9a1-4200-8965-95862c4902c9 None None] MFA Rules not processed for user `de09b91800314d06aeddafe3c12a77b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:19.865685 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-cd81d2a8-a9a1-4200-8965-95862c4902c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:19.881033 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba796b1-a467-4cdb-934d-bc0b66e9c307 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cec0bb6f1f3046b58cf9ae341de94dfd, user_id=c9644fcbc1634e38acb75492a10ca32c, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:19.881472 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 818/1659] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:29:19.891251 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba796b1-a467-4cdb-934d-bc0b66e9c307 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:19.897758 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd81d2a8-a9a1-4200-8965-95862c4902c9 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:19.902700 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-cd81d2a8-a9a1-4200-8965-95862c4902c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:19.949770 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 842/1660] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:29:19 2022] PUT /identity/v3/projects/cec0bb6f1f3046b58cf9ae341de94dfd/users/c9644fcbc1634e38acb75492a10ca32c/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:29:19.960900 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e27d6-7d9e-4a70-be31-8d7f09505b91 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:19.963578 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e27d6-7d9e-4a70-be31-8d7f09505b91 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:19.964018 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e27d6-7d9e-4a70-be31-8d7f09505b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:20.047470 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-2b4e27d6-7d9e-4a70-be31-8d7f09505b91 None None] MFA Rules not processed for user `c9644fcbc1634e38acb75492a10ca32c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:20.049120 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-2b4e27d6-7d9e-4a70-be31-8d7f09505b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:20.154170 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd81d2a8-a9a1-4200-8965-95862c4902c9 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:20.219982 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 843/1661] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:19 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:29:20.225004 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4e27d6-7d9e-4a70-be31-8d7f09505b91 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:20.281458 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:20.323701 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c62e45f-cb3c-460d-a857-267f59f36815 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:20.331766 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c62e45f-cb3c-460d-a857-267f59f36815 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:20.332198 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c62e45f-cb3c-460d-a857-267f59f36815 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:20.332360 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c62e45f-cb3c-460d-a857-267f59f36815 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:20.402439 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:20.422469 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4e27d6-7d9e-4a70-be31-8d7f09505b91 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:20.445116 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b4ed6-5ee7-44cc-b0fe-de2714d04217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:20.445408 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b4ed6-5ee7-44cc-b0fe-de2714d04217 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:20.445896 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b4ed6-5ee7-44cc-b0fe-de2714d04217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:20.446193 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b4ed6-5ee7-44cc-b0fe-de2714d04217 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:20.446961 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0b4ed6-5ee7-44cc-b0fe-de2714d04217 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:20.449080 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0b4ed6-5ee7-44cc-b0fe-de2714d04217 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:20.494934 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c62e45f-cb3c-460d-a857-267f59f36815 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:20.497739 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c62e45f-cb3c-460d-a857-267f59f36815 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:20.520743 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 819/1662] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:19 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 625 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:29:20.571696 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 844/1663] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:29:20 2022] POST /identity/v3/projects => generated 365 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:29:20.577673 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c62e45f-cb3c-460d-a857-267f59f36815 service nova] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:20.616931 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c62e45f-cb3c-460d-a857-267f59f36815 service nova] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:20.848922 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2378775-9ae9-4a49-a51b-820e2da64176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:20.849923 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2378775-9ae9-4a49-a51b-820e2da64176 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:20.850323 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2378775-9ae9-4a49-a51b-820e2da64176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:20.850788 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2378775-9ae9-4a49-a51b-820e2da64176 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:20.852177 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2378775-9ae9-4a49-a51b-820e2da64176 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:20.858805 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2378775-9ae9-4a49-a51b-820e2da64176 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:20.914900 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 820/1664] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:29:20 2022] POST /identity/v3/users => generated 289 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:20.920547 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b4ed6-5ee7-44cc-b0fe-de2714d04217 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:20.940386 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b4ed6-5ee7-44cc-b0fe-de2714d04217 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:20.952633 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01f8fea-dcad-420c-912e-48eb4c5df31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:20.953825 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01f8fea-dcad-420c-912e-48eb4c5df31f admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:20.955007 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01f8fea-dcad-420c-912e-48eb4c5df31f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:20.957388 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01f8fea-dcad-420c-912e-48eb4c5df31f admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:20.959122 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01f8fea-dcad-420c-912e-48eb4c5df31f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:20.964002 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01f8fea-dcad-420c-912e-48eb4c5df31f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:20.995171 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 845/1665] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:20 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:20.999923 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2378775-9ae9-4a49-a51b-820e2da64176 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:21.017935 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2378775-9ae9-4a49-a51b-820e2da64176 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:21.025579 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363cd45f-4b14-4ad4-954b-a5144437b743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:21.026773 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363cd45f-4b14-4ad4-954b-a5144437b743 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:21.027170 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363cd45f-4b14-4ad4-954b-a5144437b743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:21.027555 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363cd45f-4b14-4ad4-954b-a5144437b743 admin admin] PATH_INFO: `/v3/projects/09640e05f510470884282d7240d87abe/users/80d3ce8190f44b6ba9bc050b32428319/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:21.083707 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363cd45f-4b14-4ad4-954b-a5144437b743 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09640e05f510470884282d7240d87abe, user_id=80d3ce8190f44b6ba9bc050b32428319, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:21.108865 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363cd45f-4b14-4ad4-954b-a5144437b743 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:21.196652 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 821/1666] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:29:20 2022] PUT /identity/v3/projects/09640e05f510470884282d7240d87abe/users/80d3ce8190f44b6ba9bc050b32428319/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:29:21.200858 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01f8fea-dcad-420c-912e-48eb4c5df31f admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:21.246690 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01f8fea-dcad-420c-912e-48eb4c5df31f admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:21.515085 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e22d9e2-f739-4d26-854b-4c398e8ce7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:21.516245 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22d9e2-f739-4d26-854b-4c398e8ce7b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:21.516868 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22d9e2-f739-4d26-854b-4c398e8ce7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:21.517972 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22d9e2-f739-4d26-854b-4c398e8ce7b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:21.528456 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e22d9e2-f739-4d26-854b-4c398e8ce7b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:21.543749 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e22d9e2-f739-4d26-854b-4c398e8ce7b6 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:21.638569 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 846/1667] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:21 2022] GET /identity/v3/roles => generated 1420 bytes in 439 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:21.643555 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363cd45f-4b14-4ad4-954b-a5144437b743 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:21.677605 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef52c3b-f37c-457e-bf2c-5f0610d7bfa0 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:21.678397 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef52c3b-f37c-457e-bf2c-5f0610d7bfa0 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:21.679094 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef52c3b-f37c-457e-bf2c-5f0610d7bfa0 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:21.682374 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363cd45f-4b14-4ad4-954b-a5144437b743 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:21.682659 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 847/1668] 10.222.0.87 () {54 vars in 912 bytes} [Tue Jun 21 18:29:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:29:21.695836 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5117f9-88be-403f-a9f2-5e176bdacf95 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:21.696762 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5117f9-88be-403f-a9f2-5e176bdacf95 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:21.697476 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5117f9-88be-403f-a9f2-5e176bdacf95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:21.700325 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc3e54-9ef5-44a7-b790-4cfe9210567e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:21.704410 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc3e54-9ef5-44a7-b790-4cfe9210567e admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:21.704881 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc3e54-9ef5-44a7-b790-4cfe9210567e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:21.705357 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc3e54-9ef5-44a7-b790-4cfe9210567e admin admin] PATH_INFO: `/v3/projects/09640e05f510470884282d7240d87abe/users/80d3ce8190f44b6ba9bc050b32428319/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:21.709327 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fc3e54-9ef5-44a7-b790-4cfe9210567e admin admin] RBAC: Authorizing `identity:create_grant(project_id=09640e05f510470884282d7240d87abe, user_id=80d3ce8190f44b6ba9bc050b32428319, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:21.718439 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fc3e54-9ef5-44a7-b790-4cfe9210567e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:21.788614 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 822/1669] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:29:21 2022] PUT /identity/v3/projects/09640e05f510470884282d7240d87abe/users/80d3ce8190f44b6ba9bc050b32428319/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:21.814928 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a0b0f-31d9-4550-a003-ad8e159c78c5 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:21.815744 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a0b0f-31d9-4550-a003-ad8e159c78c5 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:21.816383 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a0b0f-31d9-4550-a003-ad8e159c78c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:21.849782 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-ab5117f9-88be-403f-a9f2-5e176bdacf95 None None] MFA Rules not processed for user `480e2529015c4c2fb4d6cd1985009fde`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:21.851221 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ab5117f9-88be-403f-a9f2-5e176bdacf95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:21.912644 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-380a0b0f-31d9-4550-a003-ad8e159c78c5 None None] MFA Rules not processed for user `80d3ce8190f44b6ba9bc050b32428319`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:21.915310 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-380a0b0f-31d9-4550-a003-ad8e159c78c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:22.022479 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 848/1670] 10.222.0.87 () {58 vars in 1009 bytes} [Tue Jun 21 18:29:21 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:22.119726 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:22.159652 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 823/1671] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:21 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:29:22.165288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5117f9-88be-403f-a9f2-5e176bdacf95 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:22.168365 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ab5117f9-88be-403f-a9f2-5e176bdacf95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:22.181136 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a0b0f-31d9-4550-a003-ad8e159c78c5 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:22.183736 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-380a0b0f-31d9-4550-a003-ad8e159c78c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:22.241415 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5117f9-88be-403f-a9f2-5e176bdacf95 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:22.247697 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34258e65-ad3d-4478-9a35-f79db6f08027 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:22.248581 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34258e65-ad3d-4478-9a35-f79db6f08027 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:22.249152 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34258e65-ad3d-4478-9a35-f79db6f08027 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:22.250391 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34258e65-ad3d-4478-9a35-f79db6f08027 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:22.286695 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34258e65-ad3d-4478-9a35-f79db6f08027 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:22.288846 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34258e65-ad3d-4478-9a35-f79db6f08027 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:22.329018 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 849/1672] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:29:22.382716 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a0b0f-31d9-4550-a003-ad8e159c78c5 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:22.393975 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd6a138-9e42-453e-a8e0-a0a24c15193f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:22.394594 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6a138-9e42-453e-a8e0-a0a24c15193f service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:22.394912 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6a138-9e42-453e-a8e0-a0a24c15193f service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:22.395292 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6a138-9e42-453e-a8e0-a0a24c15193f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:22.437127 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd6a138-9e42-453e-a8e0-a0a24c15193f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:22.439910 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd6a138-9e42-453e-a8e0-a0a24c15193f service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:22.452727 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 824/1673] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:22 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:29:23.715864 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34258e65-ad3d-4478-9a35-f79db6f08027 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:23.718585 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-34258e65-ad3d-4478-9a35-f79db6f08027 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:23.838993 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:24.160557 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34258e65-ad3d-4478-9a35-f79db6f08027 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:24.181124 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a474a85-f71f-4a51-bced-bfb9c0b4c9c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:24.199730 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a474a85-f71f-4a51-bced-bfb9c0b4c9c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:24.203438 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a474a85-f71f-4a51-bced-bfb9c0b4c9c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:24.206562 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a474a85-f71f-4a51-bced-bfb9c0b4c9c4 admin admin] PATH_INFO: `/v3/users/6d5ca05069584b089e2d1f26eb3eb003` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:24.217567 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a474a85-f71f-4a51-bced-bfb9c0b4c9c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d5ca05069584b089e2d1f26eb3eb003)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:24.232041 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a474a85-f71f-4a51-bced-bfb9c0b4c9c4 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:24.495162 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0a474a85-f71f-4a51-bced-bfb9c0b4c9c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d5ca05069584b089e2d1f26eb3eb003'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:24.580279 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0a474a85-f71f-4a51-bced-bfb9c0b4c9c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d5ca05069584b089e2d1f26eb3eb003'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:24.583138 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0a474a85-f71f-4a51-bced-bfb9c0b4c9c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d5ca05069584b089e2d1f26eb3eb003'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:24.587934 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0a474a85-f71f-4a51-bced-bfb9c0b4c9c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d5ca05069584b089e2d1f26eb3eb003'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:24.620044 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0a474a85-f71f-4a51-bced-bfb9c0b4c9c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d5ca05069584b089e2d1f26eb3eb003'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:24.761269 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 850/1674] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:29:23 2022] DELETE /identity/v3/users/6d5ca05069584b089e2d1f26eb3eb003 => generated 0 bytes in 1047 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:24.766874 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1a2ca-6f79-4a8a-adce-f68e6f5ac4e5 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:24.767924 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1a2ca-6f79-4a8a-adce-f68e6f5ac4e5 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:24.767924 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1a2ca-6f79-4a8a-adce-f68e6f5ac4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:24.781669 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a474a85-f71f-4a51-bced-bfb9c0b4c9c4 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:24.785210 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-0a474a85-f71f-4a51-bced-bfb9c0b4c9c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:24.820305 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-13e1a2ca-6f79-4a8a-adce-f68e6f5ac4e5 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:24.822718 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-13e1a2ca-6f79-4a8a-adce-f68e6f5ac4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:25.069349 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 825/1675] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:29:25.139410 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e1a2ca-6f79-4a8a-adce-f68e6f5ac4e5 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:25.141516 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-13e1a2ca-6f79-4a8a-adce-f68e6f5ac4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:25.298971 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a474a85-f71f-4a51-bced-bfb9c0b4c9c4 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:25.334393 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da3578-ec10-4142-850f-88deec432681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:25.336695 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da3578-ec10-4142-850f-88deec432681 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:25.337183 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da3578-ec10-4142-850f-88deec432681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:25.338508 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da3578-ec10-4142-850f-88deec432681 admin admin] PATH_INFO: `/v3/users/c9644fcbc1634e38acb75492a10ca32c` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:25.350024 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91da3578-ec10-4142-850f-88deec432681 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9644fcbc1634e38acb75492a10ca32c)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:25.361955 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91da3578-ec10-4142-850f-88deec432681 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:25.461822 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e1a2ca-6f79-4a8a-adce-f68e6f5ac4e5 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:25.473074 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc4269-78c8-48b7-ab25-553c52fe8e08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:25.474151 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc4269-78c8-48b7-ab25-553c52fe8e08 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:25.474692 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc4269-78c8-48b7-ab25-553c52fe8e08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:25.475190 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc4269-78c8-48b7-ab25-553c52fe8e08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:25.513187 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdc4269-78c8-48b7-ab25-553c52fe8e08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:25.516245 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdc4269-78c8-48b7-ab25-553c52fe8e08 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:25.555650 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 826/1676] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:29:25.577367 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:25.639011 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-91da3578-ec10-4142-850f-88deec432681 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9644fcbc1634e38acb75492a10ca32c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:25.758650 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-91da3578-ec10-4142-850f-88deec432681 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9644fcbc1634e38acb75492a10ca32c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:25.759969 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-91da3578-ec10-4142-850f-88deec432681 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9644fcbc1634e38acb75492a10ca32c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:25.761210 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-91da3578-ec10-4142-850f-88deec432681 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9644fcbc1634e38acb75492a10ca32c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:25.802135 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-91da3578-ec10-4142-850f-88deec432681 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9644fcbc1634e38acb75492a10ca32c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:25.883331 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 851/1677] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:29:24 2022] DELETE /identity/v3/users/c9644fcbc1634e38acb75492a10ca32c => generated 0 bytes in 1100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:25.886717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4ec9a6-9805-4ba9-8d7b-74a077f2dfd9 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:25.888443 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4ec9a6-9805-4ba9-8d7b-74a077f2dfd9 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:25.888931 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4ec9a6-9805-4ba9-8d7b-74a077f2dfd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:25.947544 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-aa4ec9a6-9805-4ba9-8d7b-74a077f2dfd9 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:25.949180 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-aa4ec9a6-9805-4ba9-8d7b-74a077f2dfd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:26.013610 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f45d8-be75-4ec9-8c9e-668ae1da8138 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:26.014207 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f45d8-be75-4ec9-8c9e-668ae1da8138 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:26.015667 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f45d8-be75-4ec9-8c9e-668ae1da8138 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:26.018734 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 852/1678] 10.222.0.87 () {54 vars in 960 bytes} [Tue Jun 21 18:29:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:29:26.028465 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311c85d-4921-412e-ada2-93447fd7ce60 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:26.029269 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311c85d-4921-412e-ada2-93447fd7ce60 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:26.030019 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311c85d-4921-412e-ada2-93447fd7ce60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:26.125222 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-5311c85d-4921-412e-ada2-93447fd7ce60 None None] MFA Rules not processed for user `ef1d49b36cdc4665b82487be97e9dcd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:26.131959 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5311c85d-4921-412e-ada2-93447fd7ce60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:26.146065 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:26.257381 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 827/1679] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:26.286297 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4ec9a6-9805-4ba9-8d7b-74a077f2dfd9 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:26.289035 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-aa4ec9a6-9805-4ba9-8d7b-74a077f2dfd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:26.465046 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 853/1680] 10.222.0.87 () {58 vars in 1057 bytes} [Tue Jun 21 18:29:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 439 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:26.467234 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5311c85d-4921-412e-ada2-93447fd7ce60 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:26.470655 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5311c85d-4921-412e-ada2-93447fd7ce60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:26.496640 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5311c85d-4921-412e-ada2-93447fd7ce60 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:26.499971 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4ec9a6-9805-4ba9-8d7b-74a077f2dfd9 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:26.504847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f81c5b-6cbc-45d6-a4c3-5261a16a2865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:26.506108 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f81c5b-6cbc-45d6-a4c3-5261a16a2865 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:26.506440 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f81c5b-6cbc-45d6-a4c3-5261a16a2865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:26.506738 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f81c5b-6cbc-45d6-a4c3-5261a16a2865 admin admin] PATH_INFO: `/v3/users/11342f921cf846c19bcf64211507b769` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:26.508271 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f81c5b-6cbc-45d6-a4c3-5261a16a2865 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11342f921cf846c19bcf64211507b769)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:26.510608 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac78683-8128-4346-bb3b-0474dffa6c85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:26.513097 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f81c5b-6cbc-45d6-a4c3-5261a16a2865 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:26.518469 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac78683-8128-4346-bb3b-0474dffa6c85 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:26.518791 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac78683-8128-4346-bb3b-0474dffa6c85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:26.518933 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac78683-8128-4346-bb3b-0474dffa6c85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:26.556785 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac78683-8128-4346-bb3b-0474dffa6c85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:26.560754 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac78683-8128-4346-bb3b-0474dffa6c85 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:26.587184 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 828/1681] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:29:26.590131 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac78683-8128-4346-bb3b-0474dffa6c85 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:26.670711 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-61f81c5b-6cbc-45d6-a4c3-5261a16a2865 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11342f921cf846c19bcf64211507b769'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:26.696129 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac78683-8128-4346-bb3b-0474dffa6c85 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:26.705327 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-61f81c5b-6cbc-45d6-a4c3-5261a16a2865 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11342f921cf846c19bcf64211507b769'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:26.709501 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-61f81c5b-6cbc-45d6-a4c3-5261a16a2865 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11342f921cf846c19bcf64211507b769'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:26.710674 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-61f81c5b-6cbc-45d6-a4c3-5261a16a2865 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11342f921cf846c19bcf64211507b769'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:26.723312 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25971c34-4423-48ff-9af8-f19605c88d3b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef1d49b36cdc4665b82487be97e9dcd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:26.724513 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25971c34-4423-48ff-9af8-f19605c88d3b service placement] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:26.725225 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25971c34-4423-48ff-9af8-f19605c88d3b service placement] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:26.725556 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25971c34-4423-48ff-9af8-f19605c88d3b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:26.730423 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-61f81c5b-6cbc-45d6-a4c3-5261a16a2865 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11342f921cf846c19bcf64211507b769'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:26.731024 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-25971c34-4423-48ff-9af8-f19605c88d3b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:26.761705 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 854/1682] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:29:26 2022] DELETE /identity/v3/users/11342f921cf846c19bcf64211507b769 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:26.764083 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb25e1-5a74-4b5e-bf39-84d9856a9e07 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:26.765908 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb25e1-5a74-4b5e-bf39-84d9856a9e07 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:26.767149 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb25e1-5a74-4b5e-bf39-84d9856a9e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:26.796339 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-f9cb25e1-5a74-4b5e-bf39-84d9856a9e07 None None] MFA Rules not processed for user `de09b91800314d06aeddafe3c12a77b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:26.798022 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f9cb25e1-5a74-4b5e-bf39-84d9856a9e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:26.815475 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25971c34-4423-48ff-9af8-f19605c88d3b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:26.819810 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25971c34-4423-48ff-9af8-f19605c88d3b service placement] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:26.836142 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:26.872847 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 829/1683] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:29:26.875302 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f3fcf-3a1d-48c4-acba-5c76d6599cf7 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:26.875806 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f3fcf-3a1d-48c4-acba-5c76d6599cf7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:26.877068 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f3fcf-3a1d-48c4-acba-5c76d6599cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:26.955292 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-1b1f3fcf-3a1d-48c4-acba-5c76d6599cf7 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:26.957284 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1b1f3fcf-3a1d-48c4-acba-5c76d6599cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:26.989293 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 855/1684] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:27.055646 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f13293a-00db-4a2b-8160-497083db7e97 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:27.056368 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f13293a-00db-4a2b-8160-497083db7e97 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:27.056858 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f13293a-00db-4a2b-8160-497083db7e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:27.139206 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-4f13293a-00db-4a2b-8160-497083db7e97 None None] MFA Rules not processed for user `a03657e872e94266b12d95a602038cda`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:27.140609 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4f13293a-00db-4a2b-8160-497083db7e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:27.328805 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 830/1685] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 455 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:27.379792 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f3fcf-3a1d-48c4-acba-5c76d6599cf7 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:27.381618 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1b1f3fcf-3a1d-48c4-acba-5c76d6599cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:27.436890 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 856/1686] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:29:27.448919 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac6904-bd92-400b-ba15-616b0869ced9 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:27.449227 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac6904-bd92-400b-ba15-616b0869ced9 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:27.449565 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac6904-bd92-400b-ba15-616b0869ced9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:27.530810 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-a3ac6904-bd92-400b-ba15-616b0869ced9 None None] MFA Rules not processed for user `a03657e872e94266b12d95a602038cda`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:27.532095 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a3ac6904-bd92-400b-ba15-616b0869ced9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:27.561197 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 857/1687] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:27.584323 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f3fcf-3a1d-48c4-acba-5c76d6599cf7 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:27.592538 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81dc8dd-b11e-4ec0-afa5-701b08657cab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:27.597625 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81dc8dd-b11e-4ec0-afa5-701b08657cab service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:27.598179 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81dc8dd-b11e-4ec0-afa5-701b08657cab service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:27.598617 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81dc8dd-b11e-4ec0-afa5-701b08657cab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:27.613795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ac6904-bd92-400b-ba15-616b0869ced9 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:27.616179 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a3ac6904-bd92-400b-ba15-616b0869ced9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:27.649431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81dc8dd-b11e-4ec0-afa5-701b08657cab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:27.652451 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81dc8dd-b11e-4ec0-afa5-701b08657cab service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:27.690300 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 831/1688] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:29:27.794346 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81dc8dd-b11e-4ec0-afa5-701b08657cab service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:27.796653 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b81dc8dd-b11e-4ec0-afa5-701b08657cab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:27.829572 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ac6904-bd92-400b-ba15-616b0869ced9 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:27.852287 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fd36e9-97df-494c-b4d3-c5c266cbcf91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:27.853279 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd36e9-97df-494c-b4d3-c5c266cbcf91 service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:27.853422 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd36e9-97df-494c-b4d3-c5c266cbcf91 service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:27.853873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd36e9-97df-494c-b4d3-c5c266cbcf91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:27.870143 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81dc8dd-b11e-4ec0-afa5-701b08657cab service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:27.878507 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fd36e9-97df-494c-b4d3-c5c266cbcf91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:27.880382 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fd36e9-97df-494c-b4d3-c5c266cbcf91 service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:27.884731 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aec9d3d-d83b-465d-aed2-e59f60b79d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:27.890588 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec9d3d-d83b-465d-aed2-e59f60b79d0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:27.893111 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec9d3d-d83b-465d-aed2-e59f60b79d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:27.893553 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec9d3d-d83b-465d-aed2-e59f60b79d0e admin admin] PATH_INFO: `/v3/projects/cec0bb6f1f3046b58cf9ae341de94dfd` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:27.898518 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aec9d3d-d83b-465d-aed2-e59f60b79d0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=cec0bb6f1f3046b58cf9ae341de94dfd)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:27.904634 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aec9d3d-d83b-465d-aed2-e59f60b79d0e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:27.906588 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 858/1689] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:27 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:29:27.938979 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:28.086306 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9aec9d3d-d83b-465d-aed2-e59f60b79d0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cec0bb6f1f3046b58cf9ae341de94dfd', 'request_id': 'req-9aec9d3d-d83b-465d-aed2-e59f60b79d0e'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:28.122049 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9aec9d3d-d83b-465d-aed2-e59f60b79d0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cec0bb6f1f3046b58cf9ae341de94dfd', 'request_id': 'req-9aec9d3d-d83b-465d-aed2-e59f60b79d0e'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:28.152120 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9aec9d3d-d83b-465d-aed2-e59f60b79d0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cec0bb6f1f3046b58cf9ae341de94dfd', 'request_id': 'req-9aec9d3d-d83b-465d-aed2-e59f60b79d0e'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:28.227608 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9aec9d3d-d83b-465d-aed2-e59f60b79d0e admin admin] The token cache is being invalidate because project cec0bb6f1f3046b58cf9ae341de94dfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:29:28.228239 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9aec9d3d-d83b-465d-aed2-e59f60b79d0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:28.233716 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9aec9d3d-d83b-465d-aed2-e59f60b79d0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:28.264428 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 832/1690] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:29:27 2022] DELETE /identity/v3/projects/cec0bb6f1f3046b58cf9ae341de94dfd => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:28.269332 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fd36e9-97df-494c-b4d3-c5c266cbcf91 service nova] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:28.327155 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fd36e9-97df-494c-b4d3-c5c266cbcf91 service nova] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:28.432842 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aec9d3d-d83b-465d-aed2-e59f60b79d0e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:28.435320 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9aec9d3d-d83b-465d-aed2-e59f60b79d0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:28.472277 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:28.551151 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40cf24f-b1d5-4baa-ba65-79e2b4ddcc19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:28.552784 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40cf24f-b1d5-4baa-ba65-79e2b4ddcc19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:28.554989 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40cf24f-b1d5-4baa-ba65-79e2b4ddcc19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:28.555559 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40cf24f-b1d5-4baa-ba65-79e2b4ddcc19 admin admin] PATH_INFO: `/v3/users/80d3ce8190f44b6ba9bc050b32428319` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:28.561492 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40cf24f-b1d5-4baa-ba65-79e2b4ddcc19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80d3ce8190f44b6ba9bc050b32428319)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:28.567317 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40cf24f-b1d5-4baa-ba65-79e2b4ddcc19 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:28.738408 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c40cf24f-b1d5-4baa-ba65-79e2b4ddcc19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80d3ce8190f44b6ba9bc050b32428319'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:28.807719 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c40cf24f-b1d5-4baa-ba65-79e2b4ddcc19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80d3ce8190f44b6ba9bc050b32428319'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:28.812948 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aec9d3d-d83b-465d-aed2-e59f60b79d0e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:28.822424 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c40cf24f-b1d5-4baa-ba65-79e2b4ddcc19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80d3ce8190f44b6ba9bc050b32428319'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:28.823598 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c40cf24f-b1d5-4baa-ba65-79e2b4ddcc19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80d3ce8190f44b6ba9bc050b32428319'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:28.844268 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3feae8-f141-4bfe-af84-299a135f682a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:28.844268 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3feae8-f141-4bfe-af84-299a135f682a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:28.846484 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3feae8-f141-4bfe-af84-299a135f682a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:28.847890 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3feae8-f141-4bfe-af84-299a135f682a admin admin] PATH_INFO: `/v3/projects/0940f98b5fc841d7a1b32a2a5312fd61` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:28.850801 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c40cf24f-b1d5-4baa-ba65-79e2b4ddcc19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80d3ce8190f44b6ba9bc050b32428319'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:28.854772 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3feae8-f141-4bfe-af84-299a135f682a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0940f98b5fc841d7a1b32a2a5312fd61)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:28.876243 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3feae8-f141-4bfe-af84-299a135f682a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:28.892805 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 859/1691] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:29:28 2022] DELETE /identity/v3/users/80d3ce8190f44b6ba9bc050b32428319 => generated 0 bytes in 625 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:29:29.185265 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8b3feae8-f141-4bfe-af84-299a135f682a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0940f98b5fc841d7a1b32a2a5312fd61', 'request_id': 'req-8b3feae8-f141-4bfe-af84-299a135f682a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:29.220000 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8b3feae8-f141-4bfe-af84-299a135f682a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0940f98b5fc841d7a1b32a2a5312fd61', 'request_id': 'req-8b3feae8-f141-4bfe-af84-299a135f682a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:29.232385 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8b3feae8-f141-4bfe-af84-299a135f682a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0940f98b5fc841d7a1b32a2a5312fd61', 'request_id': 'req-8b3feae8-f141-4bfe-af84-299a135f682a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:29.307703 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40cf24f-b1d5-4baa-ba65-79e2b4ddcc19 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:29.310234 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8b3feae8-f141-4bfe-af84-299a135f682a admin admin] The token cache is being invalidate because project 0940f98b5fc841d7a1b32a2a5312fd61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:29:29.310897 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8b3feae8-f141-4bfe-af84-299a135f682a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:29.311795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c40cf24f-b1d5-4baa-ba65-79e2b4ddcc19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:29.317679 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8b3feae8-f141-4bfe-af84-299a135f682a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:29.327608 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 833/1692] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:29:28 2022] DELETE /identity/v3/projects/0940f98b5fc841d7a1b32a2a5312fd61 => generated 0 bytes in 891 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:29:29.329371 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3feae8-f141-4bfe-af84-299a135f682a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:29.337904 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8b3feae8-f141-4bfe-af84-299a135f682a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:29.465345 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:29.552448 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40cf24f-b1d5-4baa-ba65-79e2b4ddcc19 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:29.592401 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3feae8-f141-4bfe-af84-299a135f682a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:29.603499 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86fb8f7-34a8-470f-9b5e-93f8edf2eb49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:29.606480 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86fb8f7-34a8-470f-9b5e-93f8edf2eb49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:29.606480 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86fb8f7-34a8-470f-9b5e-93f8edf2eb49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:29.607588 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86fb8f7-34a8-470f-9b5e-93f8edf2eb49 admin admin] PATH_INFO: `/v3/users/c4767cb6107c4a70becb08035342e66d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:29.611636 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86fb8f7-34a8-470f-9b5e-93f8edf2eb49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4767cb6107c4a70becb08035342e66d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:29.614697 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86fb8f7-34a8-470f-9b5e-93f8edf2eb49 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:29.682431 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1157d2-631a-4543-a589-ef7e4ff2fa51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:29.683584 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1157d2-631a-4543-a589-ef7e4ff2fa51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:29.683909 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1157d2-631a-4543-a589-ef7e4ff2fa51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:29.684337 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1157d2-631a-4543-a589-ef7e4ff2fa51 admin admin] PATH_INFO: `/v3/projects/a7814d8e77554771b0aad9c0768d1b91` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:29.686728 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1157d2-631a-4543-a589-ef7e4ff2fa51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7814d8e77554771b0aad9c0768d1b91)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:29.689509 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1157d2-631a-4543-a589-ef7e4ff2fa51 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:29.700624 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:29.761826 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-6e1157d2-631a-4543-a589-ef7e4ff2fa51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7814d8e77554771b0aad9c0768d1b91', 'request_id': 'req-6e1157d2-631a-4543-a589-ef7e4ff2fa51'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:29.769979 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-6e1157d2-631a-4543-a589-ef7e4ff2fa51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7814d8e77554771b0aad9c0768d1b91', 'request_id': 'req-6e1157d2-631a-4543-a589-ef7e4ff2fa51'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:29.780848 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-6e1157d2-631a-4543-a589-ef7e4ff2fa51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7814d8e77554771b0aad9c0768d1b91', 'request_id': 'req-6e1157d2-631a-4543-a589-ef7e4ff2fa51'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:29.788815 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-f86fb8f7-34a8-470f-9b5e-93f8edf2eb49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4767cb6107c4a70becb08035342e66d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:29.798778 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-6e1157d2-631a-4543-a589-ef7e4ff2fa51 admin admin] The token cache is being invalidate because project a7814d8e77554771b0aad9c0768d1b91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:29:29.799237 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-6e1157d2-631a-4543-a589-ef7e4ff2fa51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:29.800009 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-6e1157d2-631a-4543-a589-ef7e4ff2fa51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:29.802531 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 860/1693] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:29:29 2022] DELETE /identity/v3/projects/a7814d8e77554771b0aad9c0768d1b91 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:29:29.804195 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-f86fb8f7-34a8-470f-9b5e-93f8edf2eb49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4767cb6107c4a70becb08035342e66d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:29.805020 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-f86fb8f7-34a8-470f-9b5e-93f8edf2eb49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4767cb6107c4a70becb08035342e66d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:29.805814 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-f86fb8f7-34a8-470f-9b5e-93f8edf2eb49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4767cb6107c4a70becb08035342e66d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:29.824790 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1fe1e-92b7-4deb-92c1-467af033da24 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:29.825281 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1fe1e-92b7-4deb-92c1-467af033da24 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:29.825808 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1fe1e-92b7-4deb-92c1-467af033da24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:29.861601 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-f86fb8f7-34a8-470f-9b5e-93f8edf2eb49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4767cb6107c4a70becb08035342e66d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:29.906698 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-c1f1fe1e-92b7-4deb-92c1-467af033da24 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:29.910729 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 834/1694] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:29:29 2022] DELETE /identity/v3/users/c4767cb6107c4a70becb08035342e66d => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:29.911553 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c1f1fe1e-92b7-4deb-92c1-467af033da24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:30.120246 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 861/1695] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:30.141136 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1ab06-b31e-4b34-b78c-457489b91fa7 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:30.142231 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1ab06-b31e-4b34-b78c-457489b91fa7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:30.143037 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1ab06-b31e-4b34-b78c-457489b91fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:30.226550 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f9f1ab06-b31e-4b34-b78c-457489b91fa7 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:30.241584 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f9f1ab06-b31e-4b34-b78c-457489b91fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:30.351120 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 835/1696] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:29:30.353050 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1fe1e-92b7-4deb-92c1-467af033da24 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:30.421234 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1fe1e-92b7-4deb-92c1-467af033da24 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:30.488318 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2407c83b-04c7-466e-8274-d86729407dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:30.489757 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2407c83b-04c7-466e-8274-d86729407dbf admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:30.490048 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2407c83b-04c7-466e-8274-d86729407dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:30.490430 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2407c83b-04c7-466e-8274-d86729407dbf admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:30.491856 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2407c83b-04c7-466e-8274-d86729407dbf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:29:30.492362 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2407c83b-04c7-466e-8274-d86729407dbf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:30.499484 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2407c83b-04c7-466e-8274-d86729407dbf admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:30.575456 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 862/1697] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:29:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:29:30.577555 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f1ab06-b31e-4b34-b78c-457489b91fa7 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:30.614123 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f1ab06-b31e-4b34-b78c-457489b91fa7 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:30.664637 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85461f89-42eb-4e1c-8d59-e53de1d86e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:30.667480 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85461f89-42eb-4e1c-8d59-e53de1d86e48 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:30.668592 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85461f89-42eb-4e1c-8d59-e53de1d86e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:30.669106 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85461f89-42eb-4e1c-8d59-e53de1d86e48 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:30.670518 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85461f89-42eb-4e1c-8d59-e53de1d86e48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:30.674922 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85461f89-42eb-4e1c-8d59-e53de1d86e48 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:30.827104 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 836/1698] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:29:30 2022] POST /identity/v3/projects => generated 371 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:30.827444 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2407c83b-04c7-466e-8274-d86729407dbf admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:30.879194 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2407c83b-04c7-466e-8274-d86729407dbf admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:31.084753 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54c9dd-c4c6-4e2a-9911-c4876de0773b None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:31.084753 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54c9dd-c4c6-4e2a-9911-c4876de0773b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:31.085465 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54c9dd-c4c6-4e2a-9911-c4876de0773b None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:31.088610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 837/1699] 10.222.0.87 () {54 vars in 912 bytes} [Tue Jun 21 18:29:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:29:31.125245 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d0057-e5fa-4f40-9ae7-48b69ad12c52 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:31.128233 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d0057-e5fa-4f40-9ae7-48b69ad12c52 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:31.128233 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d0057-e5fa-4f40-9ae7-48b69ad12c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:31.195261 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a2b0ae-e550-4d8f-8f8e-2f684c99b74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:31.198185 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2b0ae-e550-4d8f-8f8e-2f684c99b74e admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:31.198710 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2b0ae-e550-4d8f-8f8e-2f684c99b74e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:31.203675 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2b0ae-e550-4d8f-8f8e-2f684c99b74e admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:31.209034 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a2b0ae-e550-4d8f-8f8e-2f684c99b74e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:31.213083 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a2b0ae-e550-4d8f-8f8e-2f684c99b74e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:31.229737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-961d0057-e5fa-4f40-9ae7-48b69ad12c52 None None] MFA Rules not processed for user `480e2529015c4c2fb4d6cd1985009fde`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:31.231953 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-961d0057-e5fa-4f40-9ae7-48b69ad12c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:31.293160 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 863/1700] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:29:30 2022] POST /identity/v3/users => generated 286 bytes in 467 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:29:31.296696 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a2b0ae-e550-4d8f-8f8e-2f684c99b74e admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:31.307924 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-68a2b0ae-e550-4d8f-8f8e-2f684c99b74e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:31.383548 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a2b0ae-e550-4d8f-8f8e-2f684c99b74e admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:31.404983 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52562342-beb8-4e23-969f-9399d49438d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:31.406808 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52562342-beb8-4e23-969f-9399d49438d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:31.409611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52562342-beb8-4e23-969f-9399d49438d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:31.410265 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52562342-beb8-4e23-969f-9399d49438d7 admin admin] PATH_INFO: `/v3/projects/f7881e9292d046718a487d6021876316` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:31.412864 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52562342-beb8-4e23-969f-9399d49438d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7881e9292d046718a487d6021876316)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:31.415077 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52562342-beb8-4e23-969f-9399d49438d7 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:31.480574 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:31.510215 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 838/1701] 10.222.0.87 () {58 vars in 1009 bytes} [Tue Jun 21 18:29:31 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:31.511760 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d0057-e5fa-4f40-9ae7-48b69ad12c52 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:31.555451 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d0057-e5fa-4f40-9ae7-48b69ad12c52 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:31.585463 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf194d-3354-4bd1-9a60-59921f0b292a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:31.587651 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf194d-3354-4bd1-9a60-59921f0b292a admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:31.588142 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf194d-3354-4bd1-9a60-59921f0b292a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:31.588631 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf194d-3354-4bd1-9a60-59921f0b292a admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:31.590083 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bf194d-3354-4bd1-9a60-59921f0b292a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:31.593252 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bf194d-3354-4bd1-9a60-59921f0b292a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:31.624198 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-52562342-beb8-4e23-969f-9399d49438d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7881e9292d046718a487d6021876316', 'request_id': 'req-52562342-beb8-4e23-969f-9399d49438d7'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:31.646746 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-52562342-beb8-4e23-969f-9399d49438d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7881e9292d046718a487d6021876316', 'request_id': 'req-52562342-beb8-4e23-969f-9399d49438d7'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:31.662709 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-52562342-beb8-4e23-969f-9399d49438d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7881e9292d046718a487d6021876316', 'request_id': 'req-52562342-beb8-4e23-969f-9399d49438d7'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:31.697210 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 839/1702] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:31.698881 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf194d-3354-4bd1-9a60-59921f0b292a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:31.703539 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-52562342-beb8-4e23-969f-9399d49438d7 admin admin] The token cache is being invalidate because project f7881e9292d046718a487d6021876316 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:29:31.703750 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-52562342-beb8-4e23-969f-9399d49438d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:31.705166 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-20bf194d-3354-4bd1-9a60-59921f0b292a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:31.710508 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-52562342-beb8-4e23-969f-9399d49438d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:31.712781 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 864/1703] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:29:31 2022] DELETE /identity/v3/projects/f7881e9292d046718a487d6021876316 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:31.713711 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52562342-beb8-4e23-969f-9399d49438d7 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:31.720498 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-52562342-beb8-4e23-969f-9399d49438d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:31.762092 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf194d-3354-4bd1-9a60-59921f0b292a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:31.826714 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:31.945354 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52562342-beb8-4e23-969f-9399d49438d7 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:31.949173 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae71703-21a3-412d-a64b-203c663ee851 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:31.949902 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae71703-21a3-412d-a64b-203c663ee851 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:31.950196 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae71703-21a3-412d-a64b-203c663ee851 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:31.951796 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae71703-21a3-412d-a64b-203c663ee851 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:31.953400 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-cae71703-21a3-412d-a64b-203c663ee851 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:31.975005 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132e9605-f9c5-4c78-bf94-0ee1618bb684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:31.976725 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e9605-f9c5-4c78-bf94-0ee1618bb684 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:31.980898 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e9605-f9c5-4c78-bf94-0ee1618bb684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:31.981680 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e9605-f9c5-4c78-bf94-0ee1618bb684 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:31.982711 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae71703-21a3-412d-a64b-203c663ee851 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:31.984014 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132e9605-f9c5-4c78-bf94-0ee1618bb684 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:31.984559 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae71703-21a3-412d-a64b-203c663ee851 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:31.991695 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132e9605-f9c5-4c78-bf94-0ee1618bb684 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:32.009960 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 840/1704] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:29:32.012097 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae71703-21a3-412d-a64b-203c663ee851 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:32.019491 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-cae71703-21a3-412d-a64b-203c663ee851 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:32.073480 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 865/1705] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:32.077292 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae71703-21a3-412d-a64b-203c663ee851 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:32.078782 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba8565f-bd1e-44ac-9758-fab5b6ec5998 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:32.079987 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba8565f-bd1e-44ac-9758-fab5b6ec5998 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:32.080481 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba8565f-bd1e-44ac-9758-fab5b6ec5998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:32.092050 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e01067-2182-452f-826b-ec47c055f6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:32.093025 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e01067-2182-452f-826b-ec47c055f6a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:32.093339 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e01067-2182-452f-826b-ec47c055f6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:32.093581 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e01067-2182-452f-826b-ec47c055f6a3 admin admin] PATH_INFO: `/v3/users/0b750ad5e96c42bc8a5be6b3e3dd6ab5` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:32.098771 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e01067-2182-452f-826b-ec47c055f6a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b750ad5e96c42bc8a5be6b3e3dd6ab5)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:32.102158 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e01067-2182-452f-826b-ec47c055f6a3 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:32.173120 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-aba8565f-bd1e-44ac-9758-fab5b6ec5998 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:32.175322 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-aba8565f-bd1e-44ac-9758-fab5b6ec5998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:32.222851 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-10e01067-2182-452f-826b-ec47c055f6a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b750ad5e96c42bc8a5be6b3e3dd6ab5'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:32.236259 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-10e01067-2182-452f-826b-ec47c055f6a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b750ad5e96c42bc8a5be6b3e3dd6ab5'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:32.237756 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-10e01067-2182-452f-826b-ec47c055f6a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b750ad5e96c42bc8a5be6b3e3dd6ab5'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:32.238759 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-10e01067-2182-452f-826b-ec47c055f6a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b750ad5e96c42bc8a5be6b3e3dd6ab5'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:32.250858 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 866/1706] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:32.253677 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54cb43-1886-4d74-8b68-092dae705d4d None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:32.253677 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54cb43-1886-4d74-8b68-092dae705d4d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:32.254136 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54cb43-1886-4d74-8b68-092dae705d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:32.279921 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-10e01067-2182-452f-826b-ec47c055f6a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b750ad5e96c42bc8a5be6b3e3dd6ab5'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:32.340789 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 841/1707] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:29:32 2022] DELETE /identity/v3/users/0b750ad5e96c42bc8a5be6b3e3dd6ab5 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:32.342102 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e01067-2182-452f-826b-ec47c055f6a3 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:32.345070 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-10e01067-2182-452f-826b-ec47c055f6a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:32.376865 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-5c54cb43-1886-4d74-8b68-092dae705d4d None None] MFA Rules not processed for user `09c9fa7dccb8491c9ad3b43967345ea2`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:32.386503 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5c54cb43-1886-4d74-8b68-092dae705d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:32.592030 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 867/1708] 10.222.0.87 () {58 vars in 1009 bytes} [Tue Jun 21 18:29:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:32.593172 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f694f7f-9107-48eb-842a-0c81f0fa3ecc None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:32.593466 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f694f7f-9107-48eb-842a-0c81f0fa3ecc None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:32.593778 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f694f7f-9107-48eb-842a-0c81f0fa3ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:32.635486 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e01067-2182-452f-826b-ec47c055f6a3 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:32.672272 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e7d866-3ed1-4672-b26a-9cfd135637d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:32.674620 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e7d866-3ed1-4672-b26a-9cfd135637d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:32.675268 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e7d866-3ed1-4672-b26a-9cfd135637d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:32.678725 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-9f694f7f-9107-48eb-842a-0c81f0fa3ecc None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:32.680043 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-9f694f7f-9107-48eb-842a-0c81f0fa3ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:32.684264 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e7d866-3ed1-4672-b26a-9cfd135637d3 admin admin] PATH_INFO: `/v3/projects/84f9e28ba9774ba4af4e255b84ecf74c/users/62fa006ccf304457abe457dd688386ce/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:32.711906 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 868/1709] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:32.713076 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a1681-1346-4f23-b248-7a2380203dc2 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:32.713369 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a1681-1346-4f23-b248-7a2380203dc2 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:32.713668 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a1681-1346-4f23-b248-7a2380203dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:32.750278 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e7d866-3ed1-4672-b26a-9cfd135637d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84f9e28ba9774ba4af4e255b84ecf74c, user_id=62fa006ccf304457abe457dd688386ce, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:32.755892 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e7d866-3ed1-4672-b26a-9cfd135637d3 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:32.765952 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-c46a1681-1346-4f23-b248-7a2380203dc2 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:32.767535 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c46a1681-1346-4f23-b248-7a2380203dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:32.792014 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 869/1710] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:32.793367 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46a1681-1346-4f23-b248-7a2380203dc2 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:32.807225 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 842/1711] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:29:32 2022] PUT /identity/v3/projects/84f9e28ba9774ba4af4e255b84ecf74c/users/62fa006ccf304457abe457dd688386ce/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:32.829916 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d148445f-f9fc-4ce9-9bdb-6ca922289744 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:32.830144 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d148445f-f9fc-4ce9-9bdb-6ca922289744 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:32.830417 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d148445f-f9fc-4ce9-9bdb-6ca922289744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:32.842872 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46a1681-1346-4f23-b248-7a2380203dc2 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:32.881117 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:33.007861 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfab99f9-4adb-4a7e-8074-c922315b078d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:33.008579 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab99f9-4adb-4a7e-8074-c922315b078d admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:33.008905 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab99f9-4adb-4a7e-8074-c922315b078d admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:33.011492 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab99f9-4adb-4a7e-8074-c922315b078d admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:33.012349 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfab99f9-4adb-4a7e-8074-c922315b078d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:29:33.012678 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfab99f9-4adb-4a7e-8074-c922315b078d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:33.018209 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfab99f9-4adb-4a7e-8074-c922315b078d admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:33.049942 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d148445f-f9fc-4ce9-9bdb-6ca922289744 None None] MFA Rules not processed for user `62fa006ccf304457abe457dd688386ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:33.052120 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 870/1712] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:29:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:29:33.053305 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfab99f9-4adb-4a7e-8074-c922315b078d admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:33.055516 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d148445f-f9fc-4ce9-9bdb-6ca922289744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:33.070022 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-bfab99f9-4adb-4a7e-8074-c922315b078d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:33.131905 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:33.219497 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfab99f9-4adb-4a7e-8074-c922315b078d admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:33.229827 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af52f4-54a3-4a72-8eeb-32d400a95972 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:33.230512 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af52f4-54a3-4a72-8eeb-32d400a95972 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:33.230799 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af52f4-54a3-4a72-8eeb-32d400a95972 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:33.231077 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af52f4-54a3-4a72-8eeb-32d400a95972 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:33.274621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6af52f4-54a3-4a72-8eeb-32d400a95972 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:33.277056 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6af52f4-54a3-4a72-8eeb-32d400a95972 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:33.293909 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 871/1713] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:29:33.295185 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af52f4-54a3-4a72-8eeb-32d400a95972 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:33.299382 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d6af52f4-54a3-4a72-8eeb-32d400a95972 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:33.336955 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 843/1714] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 516 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:33.344744 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d148445f-f9fc-4ce9-9bdb-6ca922289744 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:33.377236 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d148445f-f9fc-4ce9-9bdb-6ca922289744 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:33.390564 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc2264d-e6e4-4845-8297-8d260a3842a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:33.391632 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2264d-e6e4-4845-8297-8d260a3842a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:33.393375 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2264d-e6e4-4845-8297-8d260a3842a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:33.393833 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2264d-e6e4-4845-8297-8d260a3842a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:33.395382 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc2264d-e6e4-4845-8297-8d260a3842a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:33.403825 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc2264d-e6e4-4845-8297-8d260a3842a4 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:33.494356 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 844/1715] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:29:33 2022] POST /identity/v3/projects => generated 369 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:33.494356 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc2264d-e6e4-4845-8297-8d260a3842a4 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:33.510123 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af52f4-54a3-4a72-8eeb-32d400a95972 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:33.639133 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:33.667593 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3454618-8f7d-4376-b725-9aaac3fac096 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '515b3571045e4fe29565307c8ecc10b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:33.668281 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3454618-8f7d-4376-b725-9aaac3fac096 service swift] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:33.668574 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3454618-8f7d-4376-b725-9aaac3fac096 service swift] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:33.668859 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3454618-8f7d-4376-b725-9aaac3fac096 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:33.793681 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc2264d-e6e4-4845-8297-8d260a3842a4 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:33.804736 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e28401-5020-4c2e-8016-6ef450925a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:33.805433 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e28401-5020-4c2e-8016-6ef450925a2b admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:33.806084 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e28401-5020-4c2e-8016-6ef450925a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:33.806587 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e28401-5020-4c2e-8016-6ef450925a2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:33.813048 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e28401-5020-4c2e-8016-6ef450925a2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:33.813048 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e28401-5020-4c2e-8016-6ef450925a2b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:33.899848 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3454618-8f7d-4376-b725-9aaac3fac096 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:33.902250 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 845/1716] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:29:33 2022] POST /identity/v3/projects => generated 373 bytes in 412 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:33.903220 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e28401-5020-4c2e-8016-6ef450925a2b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:33.911015 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3454618-8f7d-4376-b725-9aaac3fac096 service swift] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:33.935079 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 872/1717] 10.222.0.87 () {60 vars in 1384 bytes} [Tue Jun 21 18:29:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 640 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 18:29:33.937335 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3454618-8f7d-4376-b725-9aaac3fac096 service swift] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:34.005701 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3454618-8f7d-4376-b725-9aaac3fac096 service swift] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:34.076429 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e28401-5020-4c2e-8016-6ef450925a2b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:34.107209 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b872a-fade-4b1d-bfbe-e4d214560e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:34.107947 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b872a-fade-4b1d-bfbe-e4d214560e96 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:34.108550 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b872a-fade-4b1d-bfbe-e4d214560e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:34.108835 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b872a-fade-4b1d-bfbe-e4d214560e96 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:34.109804 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88b872a-fade-4b1d-bfbe-e4d214560e96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:34.112088 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88b872a-fade-4b1d-bfbe-e4d214560e96 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:34.180874 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:34.224033 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 846/1718] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:29:33 2022] POST /identity/v3/users => generated 285 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:34.230659 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b872a-fade-4b1d-bfbe-e4d214560e96 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:34.280054 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b872a-fade-4b1d-bfbe-e4d214560e96 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:34.313733 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832be748-409c-40c0-88e7-53adcad954ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:34.314975 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832be748-409c-40c0-88e7-53adcad954ad admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:34.315465 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832be748-409c-40c0-88e7-53adcad954ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:34.315936 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832be748-409c-40c0-88e7-53adcad954ad admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:34.317240 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832be748-409c-40c0-88e7-53adcad954ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:34.326884 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620320c-6e3d-4322-8923-f9c615fd491e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:34.328202 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620320c-6e3d-4322-8923-f9c615fd491e admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:34.334655 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832be748-409c-40c0-88e7-53adcad954ad admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:34.335744 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620320c-6e3d-4322-8923-f9c615fd491e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:34.336395 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620320c-6e3d-4322-8923-f9c615fd491e admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:34.338067 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1620320c-6e3d-4322-8923-f9c615fd491e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:34.341437 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1620320c-6e3d-4322-8923-f9c615fd491e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:34.431418 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 847/1719] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:29:34.433342 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620320c-6e3d-4322-8923-f9c615fd491e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:34.438781 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 873/1720] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:29:33 2022] POST /identity/v3/users => generated 293 bytes in 503 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:34.443030 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832be748-409c-40c0-88e7-53adcad954ad admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:34.487933 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832be748-409c-40c0-88e7-53adcad954ad admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:34.507946 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620320c-6e3d-4322-8923-f9c615fd491e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:34.520548 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d61c50-4a9d-49ef-b939-8d8eb3ab13f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:34.526650 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d61c50-4a9d-49ef-b939-8d8eb3ab13f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:34.529941 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d61c50-4a9d-49ef-b939-8d8eb3ab13f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:34.536646 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d61c50-4a9d-49ef-b939-8d8eb3ab13f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:34.539587 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d61c50-4a9d-49ef-b939-8d8eb3ab13f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:34.541188 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc495de1-782e-4584-8d2c-7db9925b45e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:34.544210 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d61c50-4a9d-49ef-b939-8d8eb3ab13f8 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:34.544810 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc495de1-782e-4584-8d2c-7db9925b45e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:34.546365 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc495de1-782e-4584-8d2c-7db9925b45e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:34.557933 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc495de1-782e-4584-8d2c-7db9925b45e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:34.563407 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc495de1-782e-4584-8d2c-7db9925b45e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:34.569966 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc495de1-782e-4584-8d2c-7db9925b45e8 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:34.587723 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 874/1721] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:34.594133 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d61c50-4a9d-49ef-b939-8d8eb3ab13f8 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:34.640598 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d61c50-4a9d-49ef-b939-8d8eb3ab13f8 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:34.659887 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 848/1722] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:34.663139 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60f747-3e0d-4f26-9102-97de98b96a27 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:34.668972 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60f747-3e0d-4f26-9102-97de98b96a27 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:34.669706 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60f747-3e0d-4f26-9102-97de98b96a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:34.673257 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d9959b-1abe-491a-8121-f193a03c4931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:34.675158 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9959b-1abe-491a-8121-f193a03c4931 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:34.675698 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9959b-1abe-491a-8121-f193a03c4931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:34.676012 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9959b-1abe-491a-8121-f193a03c4931 admin admin] PATH_INFO: `/v3/projects/a399367587c2486f86f2039e5c3ebd58/users/c7ce78c48f394382a0493e7ea6d1971b/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:34.759253 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d9959b-1abe-491a-8121-f193a03c4931 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a399367587c2486f86f2039e5c3ebd58, user_id=c7ce78c48f394382a0493e7ea6d1971b, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:34.772719 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d9959b-1abe-491a-8121-f193a03c4931 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:34.786702 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-7b60f747-3e0d-4f26-9102-97de98b96a27 None None] MFA Rules not processed for user `480e2529015c4c2fb4d6cd1985009fde`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:34.791967 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7b60f747-3e0d-4f26-9102-97de98b96a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:34.828437 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 875/1723] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:29:34 2022] PUT /identity/v3/projects/a399367587c2486f86f2039e5c3ebd58/users/c7ce78c48f394382a0493e7ea6d1971b/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:34.833515 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d9959b-1abe-491a-8121-f193a03c4931 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:35.149516 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 849/1724] 10.222.0.75 () {58 vars in 1015 bytes} [Tue Jun 21 18:29:34 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 489 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:35.153788 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b60f747-3e0d-4f26-9102-97de98b96a27 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:35.204084 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b60f747-3e0d-4f26-9102-97de98b96a27 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:35.209588 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:35.280632 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d9959b-1abe-491a-8121-f193a03c4931 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:35.309082 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884877e7-6e7a-4251-ad5d-a3b566708fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:35.311005 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884877e7-6e7a-4251-ad5d-a3b566708fcc admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:35.316192 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884877e7-6e7a-4251-ad5d-a3b566708fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:35.317231 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884877e7-6e7a-4251-ad5d-a3b566708fcc admin admin] PATH_INFO: `/v3/projects/9095e82976da42d9bcc5f140a0b176d5/users/8a7ffed1a8804a7f9ebfecabf76a6063/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:35.396656 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884877e7-6e7a-4251-ad5d-a3b566708fcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9095e82976da42d9bcc5f140a0b176d5, user_id=8a7ffed1a8804a7f9ebfecabf76a6063, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:35.408826 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884877e7-6e7a-4251-ad5d-a3b566708fcc admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:35.493899 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 876/1725] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:29:34 2022] PUT /identity/v3/projects/9095e82976da42d9bcc5f140a0b176d5/users/8a7ffed1a8804a7f9ebfecabf76a6063/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 657 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:35.506117 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884877e7-6e7a-4251-ad5d-a3b566708fcc admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:35.657282 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cbc393-8614-4c91-b126-67f20b0405ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:35.665183 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cbc393-8614-4c91-b126-67f20b0405ad admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:35.684128 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cbc393-8614-4c91-b126-67f20b0405ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:35.684919 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cbc393-8614-4c91-b126-67f20b0405ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:35.709385 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cbc393-8614-4c91-b126-67f20b0405ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:35.715190 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cbc393-8614-4c91-b126-67f20b0405ad admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:35.776164 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 850/1726] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 626 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:35.782629 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f3559-beef-4be0-af58-8bd0bf4bbf81 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:35.786672 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f3559-beef-4be0-af58-8bd0bf4bbf81 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:35.787140 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f3559-beef-4be0-af58-8bd0bf4bbf81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:35.984763 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884877e7-6e7a-4251-ad5d-a3b566708fcc admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:36.007493 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-395f3559-beef-4be0-af58-8bd0bf4bbf81 None None] MFA Rules not processed for user `8a7ffed1a8804a7f9ebfecabf76a6063`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:36.013496 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6033221e-1fc0-403b-89eb-8c9aa9e5ccb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:36.022863 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033221e-1fc0-403b-89eb-8c9aa9e5ccb5 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:36.028316 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-395f3559-beef-4be0-af58-8bd0bf4bbf81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:36.044878 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033221e-1fc0-403b-89eb-8c9aa9e5ccb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:36.045525 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033221e-1fc0-403b-89eb-8c9aa9e5ccb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:36.092406 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6033221e-1fc0-403b-89eb-8c9aa9e5ccb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:36.107864 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6033221e-1fc0-403b-89eb-8c9aa9e5ccb5 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:36.183555 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 877/1727] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 687 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:29:36.185178 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6033221e-1fc0-403b-89eb-8c9aa9e5ccb5 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:36.624001 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 851/1728] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:35 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 847 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:36.630816 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f3559-beef-4be0-af58-8bd0bf4bbf81 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:36.639219 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-395f3559-beef-4be0-af58-8bd0bf4bbf81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:36.676193 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f3559-beef-4be0-af58-8bd0bf4bbf81 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:36.719176 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6033221e-1fc0-403b-89eb-8c9aa9e5ccb5 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:36.739437 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a291e6-7d22-4b1d-9d77-205fcbba3e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:36.743009 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a102c1e-4f1f-4871-aa95-7256de7d71f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:36.744713 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a102c1e-4f1f-4871-aa95-7256de7d71f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:36.745179 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a102c1e-4f1f-4871-aa95-7256de7d71f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:36.745673 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a102c1e-4f1f-4871-aa95-7256de7d71f6 admin admin] PATH_INFO: `/v3/projects/a399367587c2486f86f2039e5c3ebd58/users/c7ce78c48f394382a0493e7ea6d1971b/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:36.752035 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a291e6-7d22-4b1d-9d77-205fcbba3e7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:36.754053 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a291e6-7d22-4b1d-9d77-205fcbba3e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:36.755223 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a291e6-7d22-4b1d-9d77-205fcbba3e7f admin admin] PATH_INFO: `/v3/projects/09640e05f510470884282d7240d87abe` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:36.758737 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a102c1e-4f1f-4871-aa95-7256de7d71f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a399367587c2486f86f2039e5c3ebd58, user_id=c7ce78c48f394382a0493e7ea6d1971b, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:36.763383 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a291e6-7d22-4b1d-9d77-205fcbba3e7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=09640e05f510470884282d7240d87abe)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:36.776045 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a102c1e-4f1f-4871-aa95-7256de7d71f6 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:36.779472 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a291e6-7d22-4b1d-9d77-205fcbba3e7f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:36.827580 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 878/1729] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:29:36 2022] PUT /identity/v3/projects/a399367587c2486f86f2039e5c3ebd58/users/c7ce78c48f394382a0493e7ea6d1971b/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 643 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:36.853899 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a102c1e-4f1f-4871-aa95-7256de7d71f6 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:36.884142 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:37.103756 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-10a291e6-7d22-4b1d-9d77-205fcbba3e7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09640e05f510470884282d7240d87abe', 'request_id': 'req-10a291e6-7d22-4b1d-9d77-205fcbba3e7f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:37.121924 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-10a291e6-7d22-4b1d-9d77-205fcbba3e7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09640e05f510470884282d7240d87abe', 'request_id': 'req-10a291e6-7d22-4b1d-9d77-205fcbba3e7f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:37.138113 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a102c1e-4f1f-4871-aa95-7256de7d71f6 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:37.138945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-10a291e6-7d22-4b1d-9d77-205fcbba3e7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09640e05f510470884282d7240d87abe', 'request_id': 'req-10a291e6-7d22-4b1d-9d77-205fcbba3e7f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:37.180342 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-10a291e6-7d22-4b1d-9d77-205fcbba3e7f admin admin] The token cache is being invalidate because project 09640e05f510470884282d7240d87abe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:29:37.181961 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-10a291e6-7d22-4b1d-9d77-205fcbba3e7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:37.185480 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-10a291e6-7d22-4b1d-9d77-205fcbba3e7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:37.190639 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:37.194079 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 852/1730] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:29:36 2022] DELETE /identity/v3/projects/09640e05f510470884282d7240d87abe => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:37.200725 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78834266-a277-45c9-a8d3-54b49cdd5dde None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:37.201781 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78834266-a277-45c9-a8d3-54b49cdd5dde None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:37.202482 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78834266-a277-45c9-a8d3-54b49cdd5dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:37.293478 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-78834266-a277-45c9-a8d3-54b49cdd5dde None None] MFA Rules not processed for user `c7ce78c48f394382a0493e7ea6d1971b`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:37.297663 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-78834266-a277-45c9-a8d3-54b49cdd5dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:37.368061 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42330fa-2e1d-4324-988c-c992405fcfd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:37.369761 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42330fa-2e1d-4324-988c-c992405fcfd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:37.370367 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42330fa-2e1d-4324-988c-c992405fcfd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:37.371545 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42330fa-2e1d-4324-988c-c992405fcfd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:37.373313 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42330fa-2e1d-4324-988c-c992405fcfd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:37.376137 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42330fa-2e1d-4324-988c-c992405fcfd9 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:37.421380 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 879/1731] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:29:36 2022] POST /identity/v3/projects => generated 369 bytes in 593 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:37.423499 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a650476e-49c8-45bc-8e69-0a4f802e7fa6 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:37.424079 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a650476e-49c8-45bc-8e69-0a4f802e7fa6 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:37.424942 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a650476e-49c8-45bc-8e69-0a4f802e7fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:37.467201 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-a650476e-49c8-45bc-8e69-0a4f802e7fa6 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:37.469688 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a650476e-49c8-45bc-8e69-0a4f802e7fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:37.684664 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 853/1732] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:37 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 490 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:37.692860 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78834266-a277-45c9-a8d3-54b49cdd5dde None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:37.701248 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-78834266-a277-45c9-a8d3-54b49cdd5dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:37.788043 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78834266-a277-45c9-a8d3-54b49cdd5dde None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:37.872340 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 880/1733] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 449 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:37.889994 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a650476e-49c8-45bc-8e69-0a4f802e7fa6 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:37.893021 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a650476e-49c8-45bc-8e69-0a4f802e7fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:38.105820 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3642e5-8b0a-4e5b-b319-1c76b8b8ef7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:38.106588 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3642e5-8b0a-4e5b-b319-1c76b8b8ef7d admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:38.106904 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3642e5-8b0a-4e5b-b319-1c76b8b8ef7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:38.115090 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3642e5-8b0a-4e5b-b319-1c76b8b8ef7d admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:38.116936 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3642e5-8b0a-4e5b-b319-1c76b8b8ef7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:38.121070 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3642e5-8b0a-4e5b-b319-1c76b8b8ef7d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:38.174864 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a650476e-49c8-45bc-8e69-0a4f802e7fa6 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:38.180303 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c22182f-ae1c-413f-aad4-cf27842e7e87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:38.181190 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c22182f-ae1c-413f-aad4-cf27842e7e87 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:38.181549 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c22182f-ae1c-413f-aad4-cf27842e7e87 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:38.182069 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c22182f-ae1c-413f-aad4-cf27842e7e87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:38.202569 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 854/1734] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:29:37 2022] POST /identity/v3/users => generated 291 bytes in 509 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:38.208556 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f430cfa2-c823-46b8-a9e6-2d48e7d3f238 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:38.208985 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f430cfa2-c823-46b8-a9e6-2d48e7d3f238 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:38.209282 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f430cfa2-c823-46b8-a9e6-2d48e7d3f238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:38.277553 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f430cfa2-c823-46b8-a9e6-2d48e7d3f238 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:38.289063 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f430cfa2-c823-46b8-a9e6-2d48e7d3f238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:38.334163 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 855/1735] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:38.346104 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f430cfa2-c823-46b8-a9e6-2d48e7d3f238 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:38.352353 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f430cfa2-c823-46b8-a9e6-2d48e7d3f238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:38.369420 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:38.382866 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c22182f-ae1c-413f-aad4-cf27842e7e87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:38.390223 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c22182f-ae1c-413f-aad4-cf27842e7e87 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:38.428282 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 881/1736] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:37 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:29:38.429694 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c22182f-ae1c-413f-aad4-cf27842e7e87 service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:38.459329 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c22182f-ae1c-413f-aad4-cf27842e7e87 service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:38.483233 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b2f88e-3585-436c-a688-98b899440907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:38.487583 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2f88e-3585-436c-a688-98b899440907 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:38.488887 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2f88e-3585-436c-a688-98b899440907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:38.489026 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:38.500295 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2f88e-3585-436c-a688-98b899440907 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:38.502753 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b2f88e-3585-436c-a688-98b899440907 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:38.504073 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f430cfa2-c823-46b8-a9e6-2d48e7d3f238 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:38.515372 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b2f88e-3585-436c-a688-98b899440907 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:38.550928 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa771c92-e72f-4d1e-ab2e-16a543443eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:38.551959 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa771c92-e72f-4d1e-ab2e-16a543443eb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:38.556984 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa771c92-e72f-4d1e-ab2e-16a543443eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:38.557359 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa771c92-e72f-4d1e-ab2e-16a543443eb9 admin admin] PATH_INFO: `/v3/projects/3ec54ca89c0d4dbd822df30b3c0099fe` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:38.566494 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa771c92-e72f-4d1e-ab2e-16a543443eb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ec54ca89c0d4dbd822df30b3c0099fe)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:38.583262 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa771c92-e72f-4d1e-ab2e-16a543443eb9 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:38.591677 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 882/1737] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:38.591677 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b2f88e-3585-436c-a688-98b899440907 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:38.632948 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b2f88e-3585-436c-a688-98b899440907 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:38.658530 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a96f344-bc56-408d-8a0f-e5ed60598557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:38.662605 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96f344-bc56-408d-8a0f-e5ed60598557 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:38.663279 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96f344-bc56-408d-8a0f-e5ed60598557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:38.663800 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96f344-bc56-408d-8a0f-e5ed60598557 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:38.664843 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a96f344-bc56-408d-8a0f-e5ed60598557 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:29:38.668062 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a96f344-bc56-408d-8a0f-e5ed60598557 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:38.670126 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a96f344-bc56-408d-8a0f-e5ed60598557 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:38.730145 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 883/1738] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:29:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:29:38.733510 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a96f344-bc56-408d-8a0f-e5ed60598557 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:38.763291 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a96f344-bc56-408d-8a0f-e5ed60598557 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:38.806873 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-fa771c92-e72f-4d1e-ab2e-16a543443eb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ec54ca89c0d4dbd822df30b3c0099fe', 'request_id': 'req-fa771c92-e72f-4d1e-ab2e-16a543443eb9'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:38.827243 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-fa771c92-e72f-4d1e-ab2e-16a543443eb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ec54ca89c0d4dbd822df30b3c0099fe', 'request_id': 'req-fa771c92-e72f-4d1e-ab2e-16a543443eb9'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:38.841958 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-fa771c92-e72f-4d1e-ab2e-16a543443eb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ec54ca89c0d4dbd822df30b3c0099fe', 'request_id': 'req-fa771c92-e72f-4d1e-ab2e-16a543443eb9'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:38.886880 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-fa771c92-e72f-4d1e-ab2e-16a543443eb9 admin admin] The token cache is being invalidate because project 3ec54ca89c0d4dbd822df30b3c0099fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:29:38.887604 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-fa771c92-e72f-4d1e-ab2e-16a543443eb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:38.890744 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-fa771c92-e72f-4d1e-ab2e-16a543443eb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:38.895831 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 856/1739] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:29:38 2022] DELETE /identity/v3/projects/3ec54ca89c0d4dbd822df30b3c0099fe => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:38.898567 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa771c92-e72f-4d1e-ab2e-16a543443eb9 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:38.907421 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fa771c92-e72f-4d1e-ab2e-16a543443eb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:38.949516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbbae6a-e205-4c0e-8cd3-13ab8dd78524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:38.953725 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbae6a-e205-4c0e-8cd3-13ab8dd78524 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:38.956007 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbae6a-e205-4c0e-8cd3-13ab8dd78524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:38.956920 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbae6a-e205-4c0e-8cd3-13ab8dd78524 admin admin] PATH_INFO: `/v3/projects/961d6c911bf54d2bbccf7551e28fea16/users/cddf251debb440238621d123b586a1d7/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:39.015480 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbbae6a-e205-4c0e-8cd3-13ab8dd78524 admin admin] RBAC: Authorizing `identity:create_grant(project_id=961d6c911bf54d2bbccf7551e28fea16, user_id=cddf251debb440238621d123b586a1d7, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:39.021619 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbbae6a-e205-4c0e-8cd3-13ab8dd78524 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:39.058957 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 884/1740] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:29:38 2022] PUT /identity/v3/projects/961d6c911bf54d2bbccf7551e28fea16/users/cddf251debb440238621d123b586a1d7/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:39.062064 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbbae6a-e205-4c0e-8cd3-13ab8dd78524 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:39.065411 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-0bbbae6a-e205-4c0e-8cd3-13ab8dd78524 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:39.121341 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa771c92-e72f-4d1e-ab2e-16a543443eb9 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:39.324592 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49618fd6-0b96-428a-91a2-c53c3c4be211 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:39.325216 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49618fd6-0b96-428a-91a2-c53c3c4be211 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:39.325508 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49618fd6-0b96-428a-91a2-c53c3c4be211 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:39.325831 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49618fd6-0b96-428a-91a2-c53c3c4be211 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:39.344668 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-49618fd6-0b96-428a-91a2-c53c3c4be211 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:39.346859 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbbae6a-e205-4c0e-8cd3-13ab8dd78524 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:39.392188 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783f764f-5dc2-4280-a56b-4935c6535d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:39.396188 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f764f-5dc2-4280-a56b-4935c6535d87 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:39.396652 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f764f-5dc2-4280-a56b-4935c6535d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:39.397171 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f764f-5dc2-4280-a56b-4935c6535d87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:39.405355 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783f764f-5dc2-4280-a56b-4935c6535d87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:39.420463 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783f764f-5dc2-4280-a56b-4935c6535d87 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:39.653885 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 885/1741] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:29:39 2022] POST /identity/v3/projects => generated 355 bytes in 595 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:39.682497 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed63ea93-c495-4428-ba85-10c738d4ce30 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:39.683009 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed63ea93-c495-4428-ba85-10c738d4ce30 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:39.702921 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed63ea93-c495-4428-ba85-10c738d4ce30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:39.786619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49618fd6-0b96-428a-91a2-c53c3c4be211 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:39.788573 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49618fd6-0b96-428a-91a2-c53c3c4be211 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:39.801739 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:39.848714 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 857/1742] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 951 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:29:39.854672 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49618fd6-0b96-428a-91a2-c53c3c4be211 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:39.863135 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-49618fd6-0b96-428a-91a2-c53c3c4be211 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:39.956606 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49618fd6-0b96-428a-91a2-c53c3c4be211 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:40.217119 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4d75e4-e59f-434a-ae14-eee9d9c87fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:40.222318 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d75e4-e59f-434a-ae14-eee9d9c87fa9 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:40.222729 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d75e4-e59f-434a-ae14-eee9d9c87fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:40.229590 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d75e4-e59f-434a-ae14-eee9d9c87fa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:40.234016 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4d75e4-e59f-434a-ae14-eee9d9c87fa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:40.242196 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4d75e4-e59f-434a-ae14-eee9d9c87fa9 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:40.300712 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 858/1743] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 451 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:40.305910 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4d75e4-e59f-434a-ae14-eee9d9c87fa9 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:40.347906 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4d75e4-e59f-434a-ae14-eee9d9c87fa9 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:40.365149 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e72474-1856-4470-a671-088d283b61d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:40.367107 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e72474-1856-4470-a671-088d283b61d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:40.368259 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e72474-1856-4470-a671-088d283b61d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:40.370956 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e72474-1856-4470-a671-088d283b61d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:40.373168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e72474-1856-4470-a671-088d283b61d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:40.379502 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e72474-1856-4470-a671-088d283b61d0 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:40.435942 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 859/1744] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:29:40 2022] POST /identity/v3/users => generated 278 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:40.435942 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e72474-1856-4470-a671-088d283b61d0 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:40.496585 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e72474-1856-4470-a671-088d283b61d0 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:40.527490 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093913e7-d4e5-4013-8a16-d37089917907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:40.535466 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093913e7-d4e5-4013-8a16-d37089917907 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:40.536213 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093913e7-d4e5-4013-8a16-d37089917907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:40.536549 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093913e7-d4e5-4013-8a16-d37089917907 admin admin] PATH_INFO: `/v3/projects/961d6c911bf54d2bbccf7551e28fea16/users/cddf251debb440238621d123b586a1d7/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:40.547178 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093913e7-d4e5-4013-8a16-d37089917907 admin admin] RBAC: Authorizing `identity:create_grant(project_id=961d6c911bf54d2bbccf7551e28fea16, user_id=cddf251debb440238621d123b586a1d7, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:40.562341 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093913e7-d4e5-4013-8a16-d37089917907 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:40.636809 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 860/1745] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:29:40 2022] PUT /identity/v3/projects/961d6c911bf54d2bbccf7551e28fea16/users/cddf251debb440238621d123b586a1d7/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:40.643305 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093913e7-d4e5-4013-8a16-d37089917907 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:40.808902 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-ed63ea93-c495-4428-ba85-10c738d4ce30 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:40.836540 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ed63ea93-c495-4428-ba85-10c738d4ce30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:40.933944 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093913e7-d4e5-4013-8a16-d37089917907 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:40.946719 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4445a148-1ba6-4a5a-b8da-827a9eadb85c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:40.947500 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4445a148-1ba6-4a5a-b8da-827a9eadb85c admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:40.947801 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4445a148-1ba6-4a5a-b8da-827a9eadb85c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:40.948084 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4445a148-1ba6-4a5a-b8da-827a9eadb85c admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:40.948852 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4445a148-1ba6-4a5a-b8da-827a9eadb85c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:40.956877 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4445a148-1ba6-4a5a-b8da-827a9eadb85c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:40.984064 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 861/1746] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:40.987041 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1536889-943a-4aff-be97-38134aed47b3 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:40.987591 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1536889-943a-4aff-be97-38134aed47b3 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:40.987941 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1536889-943a-4aff-be97-38134aed47b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:41.028708 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a1536889-943a-4aff-be97-38134aed47b3 None None] MFA Rules not processed for user `cddf251debb440238621d123b586a1d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:41.030417 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a1536889-943a-4aff-be97-38134aed47b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:41.055309 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:41.279031 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:41.313767 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 886/1747] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 1647 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:41.317391 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed63ea93-c495-4428-ba85-10c738d4ce30 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:41.339663 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed63ea93-c495-4428-ba85-10c738d4ce30 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:41.358808 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960f33f-4b8a-4565-9719-8e0160a4b298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:41.361889 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960f33f-4b8a-4565-9719-8e0160a4b298 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:41.363413 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960f33f-4b8a-4565-9719-8e0160a4b298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:41.364909 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960f33f-4b8a-4565-9719-8e0160a4b298 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:41.366045 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 862/1748] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:40 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:41.367273 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9960f33f-4b8a-4565-9719-8e0160a4b298 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:41.371292 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f2f78-8b9a-4ab0-915b-6d813dd32dac None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:41.373104 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f2f78-8b9a-4ab0-915b-6d813dd32dac None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:41.373694 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9960f33f-4b8a-4565-9719-8e0160a4b298 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:41.374724 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f2f78-8b9a-4ab0-915b-6d813dd32dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:41.435117 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 887/1749] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:41.435117 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960f33f-4b8a-4565-9719-8e0160a4b298 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:41.439254 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-9960f33f-4b8a-4565-9719-8e0160a4b298 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:41.481015 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-7a6f2f78-8b9a-4ab0-915b-6d813dd32dac None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:41.494790 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7a6f2f78-8b9a-4ab0-915b-6d813dd32dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:41.554098 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 863/1750] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:41.555545 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f2f78-8b9a-4ab0-915b-6d813dd32dac None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:41.611047 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f2f78-8b9a-4ab0-915b-6d813dd32dac None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:41.633195 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5813a07d-5321-4442-8fee-0d58b877509a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:41.635706 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5813a07d-5321-4442-8fee-0d58b877509a admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:41.636206 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5813a07d-5321-4442-8fee-0d58b877509a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:41.636757 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5813a07d-5321-4442-8fee-0d58b877509a admin admin] PATH_INFO: `/v3/projects/42bcce2e8692420e97475ebbec0246ab/users/032838ae97784ca594e4f8ba8a97510f/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:41.669303 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960f33f-4b8a-4565-9719-8e0160a4b298 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:41.676889 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1714d9-79fa-4d8e-aa3f-f064442d4879 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:41.678002 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1714d9-79fa-4d8e-aa3f-f064442d4879 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:41.678549 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1714d9-79fa-4d8e-aa3f-f064442d4879 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:41.679007 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1714d9-79fa-4d8e-aa3f-f064442d4879 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:41.696839 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5813a07d-5321-4442-8fee-0d58b877509a admin admin] RBAC: Authorizing `identity:create_grant(project_id=42bcce2e8692420e97475ebbec0246ab, user_id=032838ae97784ca594e4f8ba8a97510f, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:41.704200 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5813a07d-5321-4442-8fee-0d58b877509a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:41.712228 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1714d9-79fa-4d8e-aa3f-f064442d4879 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:41.714273 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1714d9-79fa-4d8e-aa3f-f064442d4879 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:41.751686 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 888/1751] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:41 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:29:41.752041 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1714d9-79fa-4d8e-aa3f-f064442d4879 service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:41.764782 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 864/1752] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:29:41 2022] PUT /identity/v3/projects/42bcce2e8692420e97475ebbec0246ab/users/032838ae97784ca594e4f8ba8a97510f/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:41.776909 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67597eb-722a-4217-99f8-72a2043a14a3 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:41.777420 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67597eb-722a-4217-99f8-72a2043a14a3 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:41.778204 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67597eb-722a-4217-99f8-72a2043a14a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:41.787045 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1714d9-79fa-4d8e-aa3f-f064442d4879 service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:41.933997 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f67597eb-722a-4217-99f8-72a2043a14a3 None None] MFA Rules not processed for user `032838ae97784ca594e4f8ba8a97510f`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:41.942275 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f67597eb-722a-4217-99f8-72a2043a14a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:42.046825 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b80d47-ffa9-4768-82dc-5daa5c282813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:42.049831 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b80d47-ffa9-4768-82dc-5daa5c282813 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:42.051264 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b80d47-ffa9-4768-82dc-5daa5c282813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:42.051602 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b80d47-ffa9-4768-82dc-5daa5c282813 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:42.056582 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b80d47-ffa9-4768-82dc-5daa5c282813 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:29:42.057018 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b80d47-ffa9-4768-82dc-5daa5c282813 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:42.059356 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b80d47-ffa9-4768-82dc-5daa5c282813 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:42.108860 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 889/1753] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:29:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:29:42.118722 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b80d47-ffa9-4768-82dc-5daa5c282813 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:42.203714 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b80d47-ffa9-4768-82dc-5daa5c282813 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:42.247557 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97c6dfc-ccac-406f-bbc7-f628d281d4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:42.248804 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c6dfc-ccac-406f-bbc7-f628d281d4e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:42.249310 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c6dfc-ccac-406f-bbc7-f628d281d4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:42.249873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c6dfc-ccac-406f-bbc7-f628d281d4e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:42.251312 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97c6dfc-ccac-406f-bbc7-f628d281d4e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:42.274873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97c6dfc-ccac-406f-bbc7-f628d281d4e9 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:42.455008 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 890/1754] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:29:42 2022] POST /identity/v3/projects => generated 391 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:42.460148 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97c6dfc-ccac-406f-bbc7-f628d281d4e9 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:42.489861 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 865/1755] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:41 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 715 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:42.497372 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67597eb-722a-4217-99f8-72a2043a14a3 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:42.572961 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67597eb-722a-4217-99f8-72a2043a14a3 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:42.688052 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:42.792387 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97c6dfc-ccac-406f-bbc7-f628d281d4e9 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:42.872355 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5787eaf-5d11-4f2e-9d68-39d2cc0e6a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:42.881144 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5787eaf-5d11-4f2e-9d68-39d2cc0e6a58 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:42.885164 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5787eaf-5d11-4f2e-9d68-39d2cc0e6a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:42.889209 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5787eaf-5d11-4f2e-9d68-39d2cc0e6a58 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:42.895212 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5787eaf-5d11-4f2e-9d68-39d2cc0e6a58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:42.898200 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f28304-5881-493b-9f9c-0aa910d8fe09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:42.904421 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f28304-5881-493b-9f9c-0aa910d8fe09 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:42.904720 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5787eaf-5d11-4f2e-9d68-39d2cc0e6a58 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:42.905172 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f28304-5881-493b-9f9c-0aa910d8fe09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:42.907798 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f28304-5881-493b-9f9c-0aa910d8fe09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:42.910637 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f28304-5881-493b-9f9c-0aa910d8fe09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:42.923056 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f28304-5881-493b-9f9c-0aa910d8fe09 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:43.006668 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 866/1756] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:29:42 2022] POST /identity/v3/projects => generated 357 bytes in 509 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:43.009194 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f28304-5881-493b-9f9c-0aa910d8fe09 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:43.061568 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 891/1757] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:29:42 2022] POST /identity/v3/users => generated 296 bytes in 604 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:43.070325 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5787eaf-5d11-4f2e-9d68-39d2cc0e6a58 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:43.107638 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:43.131942 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5787eaf-5d11-4f2e-9d68-39d2cc0e6a58 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:43.279980 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f28304-5881-493b-9f9c-0aa910d8fe09 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:43.318188 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ff074e-de2e-4baf-9931-b5f3f349690d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:43.318188 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff074e-de2e-4baf-9931-b5f3f349690d admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:43.318188 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff074e-de2e-4baf-9931-b5f3f349690d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:43.318188 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff074e-de2e-4baf-9931-b5f3f349690d admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:43.318879 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ff074e-de2e-4baf-9931-b5f3f349690d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:43.336154 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ff074e-de2e-4baf-9931-b5f3f349690d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:43.438537 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 867/1758] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:29:43 2022] POST /identity/v3/users => generated 285 bytes in 429 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:43.443137 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b406e0c-fb2e-4c37-97ce-887f3a6f2b6b None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:43.443441 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b406e0c-fb2e-4c37-97ce-887f3a6f2b6b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:43.443810 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b406e0c-fb2e-4c37-97ce-887f3a6f2b6b None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:43.461574 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 868/1759] 10.222.0.87 () {54 vars in 921 bytes} [Tue Jun 21 18:29:43 2022] GET /identity => generated 270 bytes in 21 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:29:43.464280 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b406e0c-fb2e-4c37-97ce-887f3a6f2b6b None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:43.491332 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5082d-135a-4e52-9aca-ea1c12e70e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:43.493082 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5082d-135a-4e52-9aca-ea1c12e70e27 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:43.493426 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5082d-135a-4e52-9aca-ea1c12e70e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:43.493781 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5082d-135a-4e52-9aca-ea1c12e70e27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:43.494768 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb5082d-135a-4e52-9aca-ea1c12e70e27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:43.496935 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb5082d-135a-4e52-9aca-ea1c12e70e27 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:43.528661 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b406e0c-fb2e-4c37-97ce-887f3a6f2b6b None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:43.547529 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 892/1760] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 485 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:43.551534 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941a39c2-9a68-4c6d-b704-52da44f7018d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:43.552238 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941a39c2-9a68-4c6d-b704-52da44f7018d admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:43.552582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941a39c2-9a68-4c6d-b704-52da44f7018d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:43.552871 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941a39c2-9a68-4c6d-b704-52da44f7018d admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:43.554678 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5082d-135a-4e52-9aca-ea1c12e70e27 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:43.560287 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941a39c2-9a68-4c6d-b704-52da44f7018d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:43.562230 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941a39c2-9a68-4c6d-b704-52da44f7018d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:43.594489 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5082d-135a-4e52-9aca-ea1c12e70e27 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:43.608680 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 869/1761] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:43.612331 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21481b8-f04d-4179-96e6-21ab7bb6bc41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:43.612946 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941a39c2-9a68-4c6d-b704-52da44f7018d admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:43.614133 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21481b8-f04d-4179-96e6-21ab7bb6bc41 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:43.614406 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21481b8-f04d-4179-96e6-21ab7bb6bc41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:43.614750 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21481b8-f04d-4179-96e6-21ab7bb6bc41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:43.615467 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21481b8-f04d-4179-96e6-21ab7bb6bc41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:43.617556 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21481b8-f04d-4179-96e6-21ab7bb6bc41 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:43.652609 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941a39c2-9a68-4c6d-b704-52da44f7018d admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:43.663001 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 893/1762] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:43.664448 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c2ec80-67c8-45fd-ada9-2c68e225937a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:43.665441 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c2ec80-67c8-45fd-ada9-2c68e225937a admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:43.665760 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c2ec80-67c8-45fd-ada9-2c68e225937a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:43.666047 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c2ec80-67c8-45fd-ada9-2c68e225937a admin admin] PATH_INFO: `/v3/projects/67799fb01d1c464dbf5944da08298b4c/users/bdfe4d53c10a4d57844e819a239b83c3/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:43.668014 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21481b8-f04d-4179-96e6-21ab7bb6bc41 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:43.689849 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21481b8-f04d-4179-96e6-21ab7bb6bc41 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:43.700905 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5b5646-f038-4c6c-942f-3b84afbcc589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:43.704176 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b5646-f038-4c6c-942f-3b84afbcc589 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:43.704176 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b5646-f038-4c6c-942f-3b84afbcc589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:43.704841 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b5646-f038-4c6c-942f-3b84afbcc589 admin admin] PATH_INFO: `/v3/projects/8ac305317d1c4257bd02132ca784e6a8/users/09b05e5cd8f04befb99d7fc1a7412b2a/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:43.706730 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c2ec80-67c8-45fd-ada9-2c68e225937a admin admin] RBAC: Authorizing `identity:create_grant(project_id=67799fb01d1c464dbf5944da08298b4c, user_id=bdfe4d53c10a4d57844e819a239b83c3, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:43.709696 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c2ec80-67c8-45fd-ada9-2c68e225937a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:43.752868 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 870/1763] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:29:43 2022] PUT /identity/v3/projects/67799fb01d1c464dbf5944da08298b4c/users/bdfe4d53c10a4d57844e819a239b83c3/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:43.754171 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5b5646-f038-4c6c-942f-3b84afbcc589 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ac305317d1c4257bd02132ca784e6a8, user_id=09b05e5cd8f04befb99d7fc1a7412b2a, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:43.759175 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a5166-f1fc-4bc9-8926-c624ec2c787b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:43.759487 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a5166-f1fc-4bc9-8926-c624ec2c787b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:43.759905 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a5166-f1fc-4bc9-8926-c624ec2c787b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:43.764014 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5b5646-f038-4c6c-942f-3b84afbcc589 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:43.806060 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 894/1764] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:29:43 2022] PUT /identity/v3/projects/8ac305317d1c4257bd02132ca784e6a8/users/09b05e5cd8f04befb99d7fc1a7412b2a/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:43.807898 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5b5646-f038-4c6c-942f-3b84afbcc589 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:43.844963 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-8e1a5166-f1fc-4bc9-8926-c624ec2c787b None None] MFA Rules not processed for user `a1b3fa67905c4027b3656ac36e2b76c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:43.848597 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8e1a5166-f1fc-4bc9-8926-c624ec2c787b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:44.263427 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 871/1765] 10.222.0.87 () {58 vars in 1018 bytes} [Tue Jun 21 18:29:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 510 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:44.274276 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f3dc4-3a89-40c6-934f-95d4454e03ee None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:44.275829 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f3dc4-3a89-40c6-934f-95d4454e03ee None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:44.276577 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f3dc4-3a89-40c6-934f-95d4454e03ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:44.293293 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5b5646-f038-4c6c-942f-3b84afbcc589 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:44.320837 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af990b4a-b59a-4895-a1a7-94f5e53b11aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:44.322719 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af990b4a-b59a-4895-a1a7-94f5e53b11aa admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:44.327651 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af990b4a-b59a-4895-a1a7-94f5e53b11aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:44.328645 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af990b4a-b59a-4895-a1a7-94f5e53b11aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:44.334589 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af990b4a-b59a-4895-a1a7-94f5e53b11aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:44.344354 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af990b4a-b59a-4895-a1a7-94f5e53b11aa admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:44.408927 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 895/1766] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 602 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:44.414054 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af990b4a-b59a-4895-a1a7-94f5e53b11aa admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:44.430056 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-386f3dc4-3a89-40c6-934f-95d4454e03ee None None] MFA Rules not processed for user `09b05e5cd8f04befb99d7fc1a7412b2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:44.435218 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-386f3dc4-3a89-40c6-934f-95d4454e03ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:44.451716 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af990b4a-b59a-4895-a1a7-94f5e53b11aa admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:44.478003 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9cf415-0869-404b-a9a0-bfa2976b7043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:44.482981 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9cf415-0869-404b-a9a0-bfa2976b7043 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:44.498854 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9cf415-0869-404b-a9a0-bfa2976b7043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:44.500416 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9cf415-0869-404b-a9a0-bfa2976b7043 admin admin] PATH_INFO: `/v3/projects/67799fb01d1c464dbf5944da08298b4c/users/bdfe4d53c10a4d57844e819a239b83c3/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:44.515626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9cf415-0869-404b-a9a0-bfa2976b7043 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67799fb01d1c464dbf5944da08298b4c, user_id=bdfe4d53c10a4d57844e819a239b83c3, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:44.540552 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9cf415-0869-404b-a9a0-bfa2976b7043 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:44.636562 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 896/1767] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:29:44 2022] PUT /identity/v3/projects/67799fb01d1c464dbf5944da08298b4c/users/bdfe4d53c10a4d57844e819a239b83c3/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:29:44.651867 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a53d0-4f2e-46e7-a01f-d32c91a71582 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:44.652284 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a53d0-4f2e-46e7-a01f-d32c91a71582 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:44.652710 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a53d0-4f2e-46e7-a01f-d32c91a71582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:44.786239 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-151a53d0-4f2e-46e7-a01f-d32c91a71582 None None] MFA Rules not processed for user `bdfe4d53c10a4d57844e819a239b83c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:44.791563 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-151a53d0-4f2e-46e7-a01f-d32c91a71582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:44.801129 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:44.900338 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 872/1768] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:44 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 634 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:44.903910 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e4bef3-33f9-4f20-bb72-32f6679c8008 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:44.904242 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e4bef3-33f9-4f20-bb72-32f6679c8008 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:44.904499 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e4bef3-33f9-4f20-bb72-32f6679c8008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:44.957520 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:45.069408 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-19e4bef3-33f9-4f20-bb72-32f6679c8008 None None] MFA Rules not processed for user `144c9c31ed9b469aa200dcb16c89ab4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:45.070990 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-19e4bef3-33f9-4f20-bb72-32f6679c8008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:45.174025 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 897/1769] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:44 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 523 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:29:45.175856 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151a53d0-4f2e-46e7-a01f-d32c91a71582 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:45.565195 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 873/1770] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 658 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:45.565195 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e4bef3-33f9-4f20-bb72-32f6679c8008 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:45.592012 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151a53d0-4f2e-46e7-a01f-d32c91a71582 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:45.631752 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c21a2d-01d6-4bb9-9019-40573c8279a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:45.634915 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c21a2d-01d6-4bb9-9019-40573c8279a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:45.635644 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c21a2d-01d6-4bb9-9019-40573c8279a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:45.637144 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c21a2d-01d6-4bb9-9019-40573c8279a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:45.639486 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c21a2d-01d6-4bb9-9019-40573c8279a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:45.656919 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c21a2d-01d6-4bb9-9019-40573c8279a4 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:45.810285 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 898/1771] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:29:45 2022] POST /identity/v3/projects => generated 393 bytes in 635 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:45.819200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c21a2d-01d6-4bb9-9019-40573c8279a4 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:45.832357 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e4c21a2d-01d6-4bb9-9019-40573c8279a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:46.032243 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e4bef3-33f9-4f20-bb72-32f6679c8008 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:46.277158 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150d9180-2614-43bc-b864-81a3d78ef9c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:46.282001 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d9180-2614-43bc-b864-81a3d78ef9c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:46.290360 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d9180-2614-43bc-b864-81a3d78ef9c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:46.290809 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d9180-2614-43bc-b864-81a3d78ef9c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:46.314643 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c21a2d-01d6-4bb9-9019-40573c8279a4 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:46.366710 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:46.392801 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27aebd6-d903-4835-a894-15be7bcf74d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:46.394045 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27aebd6-d903-4835-a894-15be7bcf74d4 service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:46.398643 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27aebd6-d903-4835-a894-15be7bcf74d4 service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:46.399147 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27aebd6-d903-4835-a894-15be7bcf74d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:46.510153 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:46.662722 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150d9180-2614-43bc-b864-81a3d78ef9c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:46.665612 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150d9180-2614-43bc-b864-81a3d78ef9c6 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:46.677583 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27aebd6-d903-4835-a894-15be7bcf74d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:46.683874 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27aebd6-d903-4835-a894-15be7bcf74d4 service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:46.705272 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 874/1772] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:45 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 1145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:29:46.709807 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 899/1773] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 899 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:29:46.711599 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27aebd6-d903-4835-a894-15be7bcf74d4 service nova] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:46.725109 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150d9180-2614-43bc-b864-81a3d78ef9c6 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:46.732461 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-150d9180-2614-43bc-b864-81a3d78ef9c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:46.882130 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150d9180-2614-43bc-b864-81a3d78ef9c6 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:46.907935 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779bc60a-869d-4b76-84cf-4169917d9f3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:46.919741 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779bc60a-869d-4b76-84cf-4169917d9f3b service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:46.923669 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779bc60a-869d-4b76-84cf-4169917d9f3b service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:46.928077 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779bc60a-869d-4b76-84cf-4169917d9f3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:46.977460 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779bc60a-869d-4b76-84cf-4169917d9f3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:46.987688 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779bc60a-869d-4b76-84cf-4169917d9f3b service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:47.031754 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27aebd6-d903-4835-a894-15be7bcf74d4 service nova] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:47.045666 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 875/1774] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:29:47.048534 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779bc60a-869d-4b76-84cf-4169917d9f3b service nova] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:47.067797 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfca1469-d3f6-4c5a-b5f1-72bf88b31b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:47.070965 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca1469-d3f6-4c5a-b5f1-72bf88b31b4e admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:47.074610 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca1469-d3f6-4c5a-b5f1-72bf88b31b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:47.074923 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca1469-d3f6-4c5a-b5f1-72bf88b31b4e admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:47.075708 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfca1469-d3f6-4c5a-b5f1-72bf88b31b4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:47.091561 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfca1469-d3f6-4c5a-b5f1-72bf88b31b4e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:47.096482 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779bc60a-869d-4b76-84cf-4169917d9f3b service nova] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:47.112507 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df7e775-dc10-45bc-9183-9c7d877c7cf6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:47.117926 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df7e775-dc10-45bc-9183-9c7d877c7cf6 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:47.121629 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df7e775-dc10-45bc-9183-9c7d877c7cf6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:47.122176 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df7e775-dc10-45bc-9183-9c7d877c7cf6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:47.248505 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 900/1775] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:29:46 2022] POST /identity/v3/users => generated 303 bytes in 538 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:47.253248 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfca1469-d3f6-4c5a-b5f1-72bf88b31b4e admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:47.285435 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfca1469-d3f6-4c5a-b5f1-72bf88b31b4e admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:47.301446 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97730420-ea6f-4ea0-a1dc-f067e7950284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:47.303312 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97730420-ea6f-4ea0-a1dc-f067e7950284 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:47.304029 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97730420-ea6f-4ea0-a1dc-f067e7950284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:47.304922 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97730420-ea6f-4ea0-a1dc-f067e7950284 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:47.306166 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97730420-ea6f-4ea0-a1dc-f067e7950284 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:47.309101 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97730420-ea6f-4ea0-a1dc-f067e7950284 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:47.315579 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df7e775-dc10-45bc-9183-9c7d877c7cf6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:47.319073 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df7e775-dc10-45bc-9183-9c7d877c7cf6 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:47.345244 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 876/1776] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:47 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:29:47.350609 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 901/1777] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:47.354610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df7e775-dc10-45bc-9183-9c7d877c7cf6 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:47.392097 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df7e775-dc10-45bc-9183-9c7d877c7cf6 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:47.423975 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e50ec-5e8d-44d3-a568-f6bae7836d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:47.425867 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e50ec-5e8d-44d3-a568-f6bae7836d01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:47.428997 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e50ec-5e8d-44d3-a568-f6bae7836d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:47.429431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e50ec-5e8d-44d3-a568-f6bae7836d01 admin admin] PATH_INFO: `/v3/projects/92e13ef51b494757aab7bad432e979a2/users/36363726ec9d4a2fb2da57f4e83a2aa7/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:47.472615 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8471a68-c5ed-40a3-b7fc-e4531fd0c24a None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:47.472993 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8471a68-c5ed-40a3-b7fc-e4531fd0c24a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:47.473217 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8471a68-c5ed-40a3-b7fc-e4531fd0c24a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:47.514162 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702e50ec-5e8d-44d3-a568-f6bae7836d01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92e13ef51b494757aab7bad432e979a2, user_id=36363726ec9d4a2fb2da57f4e83a2aa7, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:47.526172 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702e50ec-5e8d-44d3-a568-f6bae7836d01 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:47.591995 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 877/1778] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:29:47 2022] PUT /identity/v3/projects/92e13ef51b494757aab7bad432e979a2/users/36363726ec9d4a2fb2da57f4e83a2aa7/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:47.596890 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e50ec-5e8d-44d3-a568-f6bae7836d01 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:47.604120 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-a8471a68-c5ed-40a3-b7fc-e4531fd0c24a None None] MFA Rules not processed for user `acf9a33e9fb941939010989a89a51d38`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:47.606489 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a8471a68-c5ed-40a3-b7fc-e4531fd0c24a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:47.730222 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:47.820964 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e50ec-5e8d-44d3-a568-f6bae7836d01 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:47.854301 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f241bc4b-1549-47ca-bed2-f271e500e471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:47.859152 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241bc4b-1549-47ca-bed2-f271e500e471 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:47.861042 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241bc4b-1549-47ca-bed2-f271e500e471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:47.862488 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241bc4b-1549-47ca-bed2-f271e500e471 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:47.870819 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f241bc4b-1549-47ca-bed2-f271e500e471 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:47.871073 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 902/1779] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 399 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:29:47.872594 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8471a68-c5ed-40a3-b7fc-e4531fd0c24a None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:47.877060 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f241bc4b-1549-47ca-bed2-f271e500e471 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:47.916220 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8471a68-c5ed-40a3-b7fc-e4531fd0c24a None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:47.933548 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 878/1780] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:47.938723 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f241bc4b-1549-47ca-bed2-f271e500e471 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:47.948255 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae356004-b22c-4898-9730-d29238758fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:47.954824 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae356004-b22c-4898-9730-d29238758fd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:47.955866 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae356004-b22c-4898-9730-d29238758fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:47.958329 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae356004-b22c-4898-9730-d29238758fd9 admin admin] PATH_INFO: `/v3/users/8a7ffed1a8804a7f9ebfecabf76a6063` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:47.968491 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae356004-b22c-4898-9730-d29238758fd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a7ffed1a8804a7f9ebfecabf76a6063)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:47.968792 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae356004-b22c-4898-9730-d29238758fd9 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:47.983199 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f241bc4b-1549-47ca-bed2-f271e500e471 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:48.013385 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d852a4a-1a2f-4e1c-bace-08ad5c79e0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:48.016370 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d852a4a-1a2f-4e1c-bace-08ad5c79e0cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:48.020714 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d852a4a-1a2f-4e1c-bace-08ad5c79e0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:48.021157 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d852a4a-1a2f-4e1c-bace-08ad5c79e0cd admin admin] PATH_INFO: `/v3/projects/92e13ef51b494757aab7bad432e979a2/users/36363726ec9d4a2fb2da57f4e83a2aa7/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:48.033253 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d852a4a-1a2f-4e1c-bace-08ad5c79e0cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=92e13ef51b494757aab7bad432e979a2, user_id=36363726ec9d4a2fb2da57f4e83a2aa7, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:48.048637 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d852a4a-1a2f-4e1c-bace-08ad5c79e0cd admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:48.098113 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 879/1781] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:29:47 2022] PUT /identity/v3/projects/92e13ef51b494757aab7bad432e979a2/users/36363726ec9d4a2fb2da57f4e83a2aa7/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:48.099947 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d852a4a-1a2f-4e1c-bace-08ad5c79e0cd admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:48.123169 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:48.139996 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-ae356004-b22c-4898-9730-d29238758fd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a7ffed1a8804a7f9ebfecabf76a6063'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:48.185585 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-ae356004-b22c-4898-9730-d29238758fd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a7ffed1a8804a7f9ebfecabf76a6063'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:48.187039 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-ae356004-b22c-4898-9730-d29238758fd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a7ffed1a8804a7f9ebfecabf76a6063'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:48.189535 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-ae356004-b22c-4898-9730-d29238758fd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a7ffed1a8804a7f9ebfecabf76a6063'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:48.222757 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-ae356004-b22c-4898-9730-d29238758fd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a7ffed1a8804a7f9ebfecabf76a6063'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:48.303578 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 903/1782] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:29:47 2022] DELETE /identity/v3/users/8a7ffed1a8804a7f9ebfecabf76a6063 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:48.313395 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc24af4a-1795-4aa5-a3fe-926b9797580a None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:48.313905 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc24af4a-1795-4aa5-a3fe-926b9797580a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:48.314395 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc24af4a-1795-4aa5-a3fe-926b9797580a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:48.440805 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d852a4a-1a2f-4e1c-bace-08ad5c79e0cd admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:48.466214 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-bc24af4a-1795-4aa5-a3fe-926b9797580a None None] MFA Rules not processed for user `36363726ec9d4a2fb2da57f4e83a2aa7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:48.470860 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-bc24af4a-1795-4aa5-a3fe-926b9797580a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:48.680575 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f04f4-5eda-48d1-ad27-950a036bd0c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:48.681852 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f04f4-5eda-48d1-ad27-950a036bd0c7 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:48.682342 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f04f4-5eda-48d1-ad27-950a036bd0c7 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:48.682803 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f04f4-5eda-48d1-ad27-950a036bd0c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:48.686635 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3d1f04f4-5eda-48d1-ad27-950a036bd0c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:48.756065 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 904/1783] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:48 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 452 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:48.758407 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70eb19d-f9ce-46ae-a2d6-289c34159fe4 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:48.758856 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70eb19d-f9ce-46ae-a2d6-289c34159fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:48.759324 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70eb19d-f9ce-46ae-a2d6-289c34159fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:48.850208 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-e70eb19d-f9ce-46ae-a2d6-289c34159fe4 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:48.851585 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e70eb19d-f9ce-46ae-a2d6-289c34159fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:48.921215 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:48.956414 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1f04f4-5eda-48d1-ad27-950a036bd0c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:48.963912 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1f04f4-5eda-48d1-ad27-950a036bd0c7 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:48.992773 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 905/1784] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:48.993746 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70eb19d-f9ce-46ae-a2d6-289c34159fe4 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:48.995351 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e70eb19d-f9ce-46ae-a2d6-289c34159fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:49.025095 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 880/1785] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:48 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 926 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:29:49.026970 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f04f4-5eda-48d1-ad27-950a036bd0c7 service nova] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:49.035647 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3d1f04f4-5eda-48d1-ad27-950a036bd0c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:49.162978 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70eb19d-f9ce-46ae-a2d6-289c34159fe4 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:49.168436 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f5c806-fa8e-46de-8ee3-89bc3029e2c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:49.169270 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5c806-fa8e-46de-8ee3-89bc3029e2c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:49.169715 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5c806-fa8e-46de-8ee3-89bc3029e2c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:49.170122 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5c806-fa8e-46de-8ee3-89bc3029e2c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:49.188984 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f5c806-fa8e-46de-8ee3-89bc3029e2c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:49.191272 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f5c806-fa8e-46de-8ee3-89bc3029e2c5 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:49.217577 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 906/1786] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:48 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:29:49.419238 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:49.502392 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f04f4-5eda-48d1-ad27-950a036bd0c7 service nova] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:49.512125 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f5c806-fa8e-46de-8ee3-89bc3029e2c5 service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:49.512913 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b3d7f-467a-4ed8-9bdd-a2e046eecada service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:49.514378 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b3d7f-467a-4ed8-9bdd-a2e046eecada service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:49.514595 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-80f5c806-fa8e-46de-8ee3-89bc3029e2c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:49.515301 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b3d7f-467a-4ed8-9bdd-a2e046eecada service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:49.516050 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b3d7f-467a-4ed8-9bdd-a2e046eecada service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:49.547660 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107b3d7f-467a-4ed8-9bdd-a2e046eecada service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:49.552646 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107b3d7f-467a-4ed8-9bdd-a2e046eecada service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:49.580804 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f5c806-fa8e-46de-8ee3-89bc3029e2c5 service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:49.589225 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 881/1787] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:29:49.620187 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9686a8c9-55f8-4096-928d-fc28b8a9613d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:49.621315 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686a8c9-55f8-4096-928d-fc28b8a9613d service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:49.621844 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686a8c9-55f8-4096-928d-fc28b8a9613d service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:49.622343 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686a8c9-55f8-4096-928d-fc28b8a9613d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:49.637015 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-9686a8c9-55f8-4096-928d-fc28b8a9613d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:49.984141 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9686a8c9-55f8-4096-928d-fc28b8a9613d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:49.986530 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9686a8c9-55f8-4096-928d-fc28b8a9613d service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:50.094422 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 907/1788] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:49 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 580 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:29:56.270680 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e881f7d-06c4-475f-a516-a8a4b85b4949 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:56.271563 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e881f7d-06c4-475f-a516-a8a4b85b4949 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:56.271563 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e881f7d-06c4-475f-a516-a8a4b85b4949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:56.323444 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-6e881f7d-06c4-475f-a516-a8a4b85b4949 None None] MFA Rules not processed for user `144c9c31ed9b469aa200dcb16c89ab4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:56.329616 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6e881f7d-06c4-475f-a516-a8a4b85b4949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:56.618309 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 882/1789] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:29:57.393001 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9686a8c9-55f8-4096-928d-fc28b8a9613d service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:57.395191 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-9686a8c9-55f8-4096-928d-fc28b8a9613d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:57.430927 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9686a8c9-55f8-4096-928d-fc28b8a9613d service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:57.461484 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f2f9b-cfb2-4bf6-bc65-7011b007205c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:57.462909 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f2f9b-cfb2-4bf6-bc65-7011b007205c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:57.463545 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f2f9b-cfb2-4bf6-bc65-7011b007205c admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:57.464083 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f2f9b-cfb2-4bf6-bc65-7011b007205c admin admin] PATH_INFO: `/v3/projects/9095e82976da42d9bcc5f140a0b176d5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:57.466541 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21f2f9b-cfb2-4bf6-bc65-7011b007205c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9095e82976da42d9bcc5f140a0b176d5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:57.473981 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21f2f9b-cfb2-4bf6-bc65-7011b007205c admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:57.673626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-b21f2f9b-cfb2-4bf6-bc65-7011b007205c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9095e82976da42d9bcc5f140a0b176d5', 'request_id': 'req-b21f2f9b-cfb2-4bf6-bc65-7011b007205c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:57.690281 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-b21f2f9b-cfb2-4bf6-bc65-7011b007205c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9095e82976da42d9bcc5f140a0b176d5', 'request_id': 'req-b21f2f9b-cfb2-4bf6-bc65-7011b007205c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:57.711653 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-b21f2f9b-cfb2-4bf6-bc65-7011b007205c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9095e82976da42d9bcc5f140a0b176d5', 'request_id': 'req-b21f2f9b-cfb2-4bf6-bc65-7011b007205c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:57.714963 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:57.750275 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-b21f2f9b-cfb2-4bf6-bc65-7011b007205c admin admin] The token cache is being invalidate because project 9095e82976da42d9bcc5f140a0b176d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:29:57.752649 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-b21f2f9b-cfb2-4bf6-bc65-7011b007205c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:57.760074 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-b21f2f9b-cfb2-4bf6-bc65-7011b007205c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:57.772294 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 908/1790] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:29:57 2022] DELETE /identity/v3/projects/9095e82976da42d9bcc5f140a0b176d5 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:57.774197 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e881f7d-06c4-475f-a516-a8a4b85b4949 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:57.831929 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e881f7d-06c4-475f-a516-a8a4b85b4949 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:58.078935 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3db9d0f-4c89-49e6-9e4e-9d0aff5e891a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:58.080288 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db9d0f-4c89-49e6-9e4e-9d0aff5e891a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:58.081358 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db9d0f-4c89-49e6-9e4e-9d0aff5e891a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:58.081791 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db9d0f-4c89-49e6-9e4e-9d0aff5e891a admin admin] PATH_INFO: `/v3/users/cddf251debb440238621d123b586a1d7` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:58.083667 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3db9d0f-4c89-49e6-9e4e-9d0aff5e891a admin admin] RBAC: Authorizing `identity:delete_user(user_id=cddf251debb440238621d123b586a1d7)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:58.087103 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3db9d0f-4c89-49e6-9e4e-9d0aff5e891a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:58.235810 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e3db9d0f-4c89-49e6-9e4e-9d0aff5e891a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cddf251debb440238621d123b586a1d7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:58.257339 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e3db9d0f-4c89-49e6-9e4e-9d0aff5e891a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cddf251debb440238621d123b586a1d7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:58.258782 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e3db9d0f-4c89-49e6-9e4e-9d0aff5e891a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cddf251debb440238621d123b586a1d7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:58.260749 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e3db9d0f-4c89-49e6-9e4e-9d0aff5e891a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cddf251debb440238621d123b586a1d7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:58.298554 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e3db9d0f-4c89-49e6-9e4e-9d0aff5e891a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cddf251debb440238621d123b586a1d7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:58.346028 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 883/1791] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:29:57 2022] DELETE /identity/v3/users/cddf251debb440238621d123b586a1d7 => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:29:58.969006 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f2f9b-cfb2-4bf6-bc65-7011b007205c admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:58.973132 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b21f2f9b-cfb2-4bf6-bc65-7011b007205c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:59.111845 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f2f9b-cfb2-4bf6-bc65-7011b007205c admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:59.333379 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7992d5e1-22c4-4045-996c-8c4f79c6127d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:59.334855 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992d5e1-22c4-4045-996c-8c4f79c6127d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:59.335439 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992d5e1-22c4-4045-996c-8c4f79c6127d admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:59.335921 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992d5e1-22c4-4045-996c-8c4f79c6127d admin admin] PATH_INFO: `/v3/users/09b05e5cd8f04befb99d7fc1a7412b2a` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:59.338975 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7992d5e1-22c4-4045-996c-8c4f79c6127d admin admin] RBAC: Authorizing `identity:delete_user(user_id=09b05e5cd8f04befb99d7fc1a7412b2a)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:59.344533 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7992d5e1-22c4-4045-996c-8c4f79c6127d admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:59.491173 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7992d5e1-22c4-4045-996c-8c4f79c6127d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09b05e5cd8f04befb99d7fc1a7412b2a'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:59.527473 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7992d5e1-22c4-4045-996c-8c4f79c6127d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09b05e5cd8f04befb99d7fc1a7412b2a'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:59.528867 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7992d5e1-22c4-4045-996c-8c4f79c6127d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09b05e5cd8f04befb99d7fc1a7412b2a'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:59.530410 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7992d5e1-22c4-4045-996c-8c4f79c6127d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09b05e5cd8f04befb99d7fc1a7412b2a'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:59.565102 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7992d5e1-22c4-4045-996c-8c4f79c6127d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09b05e5cd8f04befb99d7fc1a7412b2a'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:59.622148 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 909/1792] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:29:58 2022] DELETE /identity/v3/users/09b05e5cd8f04befb99d7fc1a7412b2a => generated 0 bytes in 654 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:29:59.630142 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5360317d-e0d6-4999-a459-9dbd6d7b752a None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:59.630984 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5360317d-e0d6-4999-a459-9dbd6d7b752a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:59.631128 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5360317d-e0d6-4999-a459-9dbd6d7b752a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:59.706126 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:59.739828 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-5360317d-e0d6-4999-a459-9dbd6d7b752a None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:59.742789 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5360317d-e0d6-4999-a459-9dbd6d7b752a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:59.948379 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 884/1793] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:29:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:59.974213 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7992d5e1-22c4-4045-996c-8c4f79c6127d admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:59.978539 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7992d5e1-22c4-4045-996c-8c4f79c6127d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:00.234096 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5360317d-e0d6-4999-a459-9dbd6d7b752a None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:00.234488 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7992d5e1-22c4-4045-996c-8c4f79c6127d admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:00.237667 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5360317d-e0d6-4999-a459-9dbd6d7b752a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:00.241602 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1658b7d4-a584-42e7-aeb6-dedd8b2bee24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:00.242876 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1658b7d4-a584-42e7-aeb6-dedd8b2bee24 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:00.243381 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1658b7d4-a584-42e7-aeb6-dedd8b2bee24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:00.243866 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1658b7d4-a584-42e7-aeb6-dedd8b2bee24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:00.261005 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:00.287397 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1658b7d4-a584-42e7-aeb6-dedd8b2bee24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:00.290069 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1658b7d4-a584-42e7-aeb6-dedd8b2bee24 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:00.314892 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 910/1794] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:30:00.331616 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5360317d-e0d6-4999-a459-9dbd6d7b752a None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:00.348355 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d4780-925c-4f76-8495-3833095f93d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:00.349801 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d4780-925c-4f76-8495-3833095f93d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:00.350339 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d4780-925c-4f76-8495-3833095f93d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:00.350790 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d4780-925c-4f76-8495-3833095f93d8 admin admin] PATH_INFO: `/v3/projects/961d6c911bf54d2bbccf7551e28fea16` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:00.353670 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76d4780-925c-4f76-8495-3833095f93d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=961d6c911bf54d2bbccf7551e28fea16)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:00.360460 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76d4780-925c-4f76-8495-3833095f93d8 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:00.519335 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e7c94-18e3-47cd-b51f-251974e187b3 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:00.519840 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e7c94-18e3-47cd-b51f-251974e187b3 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:00.520673 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e7c94-18e3-47cd-b51f-251974e187b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:00.527021 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d76d4780-925c-4f76-8495-3833095f93d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '961d6c911bf54d2bbccf7551e28fea16', 'request_id': 'req-d76d4780-925c-4f76-8495-3833095f93d8'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:00.561298 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d76d4780-925c-4f76-8495-3833095f93d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '961d6c911bf54d2bbccf7551e28fea16', 'request_id': 'req-d76d4780-925c-4f76-8495-3833095f93d8'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:00.574835 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d76d4780-925c-4f76-8495-3833095f93d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '961d6c911bf54d2bbccf7551e28fea16', 'request_id': 'req-d76d4780-925c-4f76-8495-3833095f93d8'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:00.604223 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d76d4780-925c-4f76-8495-3833095f93d8 admin admin] The token cache is being invalidate because project 961d6c911bf54d2bbccf7551e28fea16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:30:00.609099 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d76d4780-925c-4f76-8495-3833095f93d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:00.611487 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d76d4780-925c-4f76-8495-3833095f93d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:00.618575 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 885/1795] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:30:00 2022] DELETE /identity/v3/projects/961d6c911bf54d2bbccf7551e28fea16 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:00.619699 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-c63e7c94-18e3-47cd-b51f-251974e187b3 None None] MFA Rules not processed for user `acf9a33e9fb941939010989a89a51d38`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:00.621547 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7193e-d421-4bb9-98fa-75617e08c6e2 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:00.623658 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7193e-d421-4bb9-98fa-75617e08c6e2 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:00.624044 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7193e-d421-4bb9-98fa-75617e08c6e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:00.625939 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c63e7c94-18e3-47cd-b51f-251974e187b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:00.786927 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 911/1796] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:00.827480 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-71d7193e-d421-4bb9-98fa-75617e08c6e2 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:00.831180 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-71d7193e-d421-4bb9-98fa-75617e08c6e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:01.006018 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 886/1797] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:01.007098 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57f221-add5-4cda-a9d0-06cd3f9234c1 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:01.008607 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57f221-add5-4cda-a9d0-06cd3f9234c1 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:01.009094 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57f221-add5-4cda-a9d0-06cd3f9234c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:01.053016 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-fb57f221-add5-4cda-a9d0-06cd3f9234c1 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:01.054627 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-fb57f221-add5-4cda-a9d0-06cd3f9234c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:01.112326 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 912/1798] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:01.114980 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7193e-d421-4bb9-98fa-75617e08c6e2 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:01.151810 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7193e-d421-4bb9-98fa-75617e08c6e2 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:01.173463 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537cb3c2-e3bb-4e39-a36d-cb4b28f14184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:01.175945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537cb3c2-e3bb-4e39-a36d-cb4b28f14184 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:01.176437 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537cb3c2-e3bb-4e39-a36d-cb4b28f14184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:01.176865 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537cb3c2-e3bb-4e39-a36d-cb4b28f14184 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:01.178214 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537cb3c2-e3bb-4e39-a36d-cb4b28f14184 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:30:01.178738 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537cb3c2-e3bb-4e39-a36d-cb4b28f14184 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:01.190233 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537cb3c2-e3bb-4e39-a36d-cb4b28f14184 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:01.243876 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:01.257963 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 887/1799] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:30:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:30:01.260089 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb57f221-add5-4cda-a9d0-06cd3f9234c1 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:01.293870 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb57f221-add5-4cda-a9d0-06cd3f9234c1 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:01.308227 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b91934-3a03-4aa6-8d6f-094dfd7adb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:01.310290 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b91934-3a03-4aa6-8d6f-094dfd7adb3f admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:01.310707 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b91934-3a03-4aa6-8d6f-094dfd7adb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:01.311629 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b91934-3a03-4aa6-8d6f-094dfd7adb3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:01.313722 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b91934-3a03-4aa6-8d6f-094dfd7adb3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:01.316454 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b91934-3a03-4aa6-8d6f-094dfd7adb3f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:01.324663 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537cb3c2-e3bb-4e39-a36d-cb4b28f14184 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:01.327543 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-537cb3c2-e3bb-4e39-a36d-cb4b28f14184 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:01.406401 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537cb3c2-e3bb-4e39-a36d-cb4b28f14184 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:01.428354 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 913/1800] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:30:01 2022] POST /identity/v3/projects => generated 381 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:30:01.430219 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d4f246-a3d4-4f78-9f63-0cea0df8bdee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:01.435623 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b91934-3a03-4aa6-8d6f-094dfd7adb3f admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:01.445867 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d4f246-a3d4-4f78-9f63-0cea0df8bdee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:01.447985 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d4f246-a3d4-4f78-9f63-0cea0df8bdee admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:01.448413 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d4f246-a3d4-4f78-9f63-0cea0df8bdee admin admin] PATH_INFO: `/v3/projects/8ac305317d1c4257bd02132ca784e6a8` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:01.458942 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d4f246-a3d4-4f78-9f63-0cea0df8bdee admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ac305317d1c4257bd02132ca784e6a8)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:01.469132 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d4f246-a3d4-4f78-9f63-0cea0df8bdee admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:01.649567 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d0d4f246-a3d4-4f78-9f63-0cea0df8bdee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ac305317d1c4257bd02132ca784e6a8', 'request_id': 'req-d0d4f246-a3d4-4f78-9f63-0cea0df8bdee'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:01.664550 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b91934-3a03-4aa6-8d6f-094dfd7adb3f admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:01.675470 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d0d4f246-a3d4-4f78-9f63-0cea0df8bdee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ac305317d1c4257bd02132ca784e6a8', 'request_id': 'req-d0d4f246-a3d4-4f78-9f63-0cea0df8bdee'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:01.704305 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d0d4f246-a3d4-4f78-9f63-0cea0df8bdee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ac305317d1c4257bd02132ca784e6a8', 'request_id': 'req-d0d4f246-a3d4-4f78-9f63-0cea0df8bdee'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:01.787234 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d0d4f246-a3d4-4f78-9f63-0cea0df8bdee admin admin] The token cache is being invalidate because project 8ac305317d1c4257bd02132ca784e6a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:30:01.787234 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d0d4f246-a3d4-4f78-9f63-0cea0df8bdee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:01.790948 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d0d4f246-a3d4-4f78-9f63-0cea0df8bdee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:01.807446 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 888/1801] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:30:01 2022] DELETE /identity/v3/projects/8ac305317d1c4257bd02132ca784e6a8 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:30:01.824645 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d4f246-a3d4-4f78-9f63-0cea0df8bdee admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:01.830787 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d0d4f246-a3d4-4f78-9f63-0cea0df8bdee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:01.855998 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da45c8-5ec8-41c2-90e3-eec5b836b8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:01.860507 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da45c8-5ec8-41c2-90e3-eec5b836b8bf admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:01.860507 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da45c8-5ec8-41c2-90e3-eec5b836b8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:01.860985 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da45c8-5ec8-41c2-90e3-eec5b836b8bf admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:01.862870 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da45c8-5ec8-41c2-90e3-eec5b836b8bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:01.868208 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da45c8-5ec8-41c2-90e3-eec5b836b8bf admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:01.898139 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d4f246-a3d4-4f78-9f63-0cea0df8bdee admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:01.923182 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c39a73-4031-40e2-ac46-6e5a003150f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:01.924957 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c39a73-4031-40e2-ac46-6e5a003150f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:01.925474 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c39a73-4031-40e2-ac46-6e5a003150f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:01.926531 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c39a73-4031-40e2-ac46-6e5a003150f0 admin admin] PATH_INFO: `/v3/users/36363726ec9d4a2fb2da57f4e83a2aa7` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:01.932381 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c39a73-4031-40e2-ac46-6e5a003150f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36363726ec9d4a2fb2da57f4e83a2aa7)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:01.943247 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 914/1802] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:30:01 2022] POST /identity/v3/users => generated 291 bytes in 504 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:30:01.944277 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c39a73-4031-40e2-ac46-6e5a003150f0 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:01.944840 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da45c8-5ec8-41c2-90e3-eec5b836b8bf admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:01.947264 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-21da45c8-5ec8-41c2-90e3-eec5b836b8bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:01.995511 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da45c8-5ec8-41c2-90e3-eec5b836b8bf admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:02.013014 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46fc6f6-f1ef-4f7e-9813-7c36fa233b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:02.014176 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46fc6f6-f1ef-4f7e-9813-7c36fa233b12 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:02.014176 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46fc6f6-f1ef-4f7e-9813-7c36fa233b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:02.014594 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46fc6f6-f1ef-4f7e-9813-7c36fa233b12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:02.015681 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46fc6f6-f1ef-4f7e-9813-7c36fa233b12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:02.017528 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46fc6f6-f1ef-4f7e-9813-7c36fa233b12 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:02.023284 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:02.049879 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 915/1803] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:01 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:30:02.054771 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46fc6f6-f1ef-4f7e-9813-7c36fa233b12 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:02.084033 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46fc6f6-f1ef-4f7e-9813-7c36fa233b12 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:02.097507 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec96367c-0913-437e-9b4b-60c278b124df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:02.098856 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec96367c-0913-437e-9b4b-60c278b124df admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:02.099568 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec96367c-0913-437e-9b4b-60c278b124df admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:02.100050 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec96367c-0913-437e-9b4b-60c278b124df admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:02.102167 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec96367c-0913-437e-9b4b-60c278b124df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:02.130746 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-11c39a73-4031-40e2-ac46-6e5a003150f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36363726ec9d4a2fb2da57f4e83a2aa7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:02.158940 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec96367c-0913-437e-9b4b-60c278b124df admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:02.179337 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-11c39a73-4031-40e2-ac46-6e5a003150f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36363726ec9d4a2fb2da57f4e83a2aa7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:02.182293 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-11c39a73-4031-40e2-ac46-6e5a003150f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36363726ec9d4a2fb2da57f4e83a2aa7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:02.183563 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-11c39a73-4031-40e2-ac46-6e5a003150f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36363726ec9d4a2fb2da57f4e83a2aa7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:02.193223 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 916/1804] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:30:02.200397 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec96367c-0913-437e-9b4b-60c278b124df admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:02.202716 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ec96367c-0913-437e-9b4b-60c278b124df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:02.224823 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec96367c-0913-437e-9b4b-60c278b124df admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:02.232720 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896fe8cd-c514-44a0-8904-d17ccebff64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:02.233820 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896fe8cd-c514-44a0-8904-d17ccebff64e admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:02.234034 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896fe8cd-c514-44a0-8904-d17ccebff64e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:02.234480 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896fe8cd-c514-44a0-8904-d17ccebff64e admin admin] PATH_INFO: `/v3/projects/a1a6aa18cdc9405895e8a9072bc60ec4/users/d93c2ac7de994271843cfacca9a4d9ec/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:02.271628 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-11c39a73-4031-40e2-ac46-6e5a003150f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36363726ec9d4a2fb2da57f4e83a2aa7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:02.272500 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896fe8cd-c514-44a0-8904-d17ccebff64e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1a6aa18cdc9405895e8a9072bc60ec4, user_id=d93c2ac7de994271843cfacca9a4d9ec, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:02.278168 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896fe8cd-c514-44a0-8904-d17ccebff64e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:02.312955 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 889/1805] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:30:01 2022] DELETE /identity/v3/users/36363726ec9d4a2fb2da57f4e83a2aa7 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:02.337744 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 917/1806] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:30:02 2022] PUT /identity/v3/projects/a1a6aa18cdc9405895e8a9072bc60ec4/users/d93c2ac7de994271843cfacca9a4d9ec/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:30:02.355895 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac103b94-5127-4a07-a349-d485e8d27ec5 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:02.356123 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac103b94-5127-4a07-a349-d485e8d27ec5 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:02.356371 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac103b94-5127-4a07-a349-d485e8d27ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:02.448024 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-ac103b94-5127-4a07-a349-d485e8d27ec5 None None] MFA Rules not processed for user `d93c2ac7de994271843cfacca9a4d9ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:02.450821 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ac103b94-5127-4a07-a349-d485e8d27ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:02.600102 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:02.681856 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 890/1807] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:30:02.686019 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896fe8cd-c514-44a0-8904-d17ccebff64e admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:02.909977 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896fe8cd-c514-44a0-8904-d17ccebff64e admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:02.926481 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6586d466-41cc-4600-9ddb-f6e874fcf2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:02.928503 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586d466-41cc-4600-9ddb-f6e874fcf2e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:02.928790 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586d466-41cc-4600-9ddb-f6e874fcf2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:02.929071 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586d466-41cc-4600-9ddb-f6e874fcf2e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:02.930322 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6586d466-41cc-4600-9ddb-f6e874fcf2e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:02.934222 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6586d466-41cc-4600-9ddb-f6e874fcf2e8 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:03.015659 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 918/1808] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:30:02 2022] POST /identity/v3/projects => generated 381 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:30:03.020785 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac103b94-5127-4a07-a349-d485e8d27ec5 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:03.083720 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac103b94-5127-4a07-a349-d485e8d27ec5 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:03.412272 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4697b436-e0e1-4dd7-9eb7-21b9443156e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:03.413773 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697b436-e0e1-4dd7-9eb7-21b9443156e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:03.414242 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697b436-e0e1-4dd7-9eb7-21b9443156e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:03.414803 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697b436-e0e1-4dd7-9eb7-21b9443156e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:03.416065 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4697b436-e0e1-4dd7-9eb7-21b9443156e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:03.420903 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4697b436-e0e1-4dd7-9eb7-21b9443156e0 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:03.473696 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 891/1809] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:30:03 2022] POST /identity/v3/users => generated 297 bytes in 452 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:30:03.475351 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6586d466-41cc-4600-9ddb-f6e874fcf2e8 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:03.496235 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6586d466-41cc-4600-9ddb-f6e874fcf2e8 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:03.505544 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a210c55-d8b1-4c28-84c5-b402dd5888dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:03.507268 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a210c55-d8b1-4c28-84c5-b402dd5888dd admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:03.508000 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a210c55-d8b1-4c28-84c5-b402dd5888dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:03.508708 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a210c55-d8b1-4c28-84c5-b402dd5888dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:03.511827 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a210c55-d8b1-4c28-84c5-b402dd5888dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:03.516193 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a210c55-d8b1-4c28-84c5-b402dd5888dd admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:03.544775 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 919/1810] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:30:03.552372 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4697b436-e0e1-4dd7-9eb7-21b9443156e0 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:03.576004 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4697b436-e0e1-4dd7-9eb7-21b9443156e0 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:03.589496 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0392b48d-da41-4672-9fe6-86b91db89320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:03.591991 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0392b48d-da41-4672-9fe6-86b91db89320 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:03.592967 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0392b48d-da41-4672-9fe6-86b91db89320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:03.593629 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0392b48d-da41-4672-9fe6-86b91db89320 admin admin] PATH_INFO: `/v3/projects/ff04efdb5c144487ac8258f1cbf4c174/users/ed2ea8a03815421791c6e57c6d58e782/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:03.634581 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0392b48d-da41-4672-9fe6-86b91db89320 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff04efdb5c144487ac8258f1cbf4c174, user_id=ed2ea8a03815421791c6e57c6d58e782, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:03.641045 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0392b48d-da41-4672-9fe6-86b91db89320 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:03.683157 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 892/1811] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:30:03 2022] PUT /identity/v3/projects/ff04efdb5c144487ac8258f1cbf4c174/users/ed2ea8a03815421791c6e57c6d58e782/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:03.689474 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a210c55-d8b1-4c28-84c5-b402dd5888dd admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:03.737097 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a210c55-d8b1-4c28-84c5-b402dd5888dd admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:04.031541 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d17f221-ebfd-4b5f-b053-9ebbbda10734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:04.034636 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17f221-ebfd-4b5f-b053-9ebbbda10734 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:04.035319 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17f221-ebfd-4b5f-b053-9ebbbda10734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:04.036107 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17f221-ebfd-4b5f-b053-9ebbbda10734 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:04.038715 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d17f221-ebfd-4b5f-b053-9ebbbda10734 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:04.045294 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d17f221-ebfd-4b5f-b053-9ebbbda10734 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:04.046797 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0392b48d-da41-4672-9fe6-86b91db89320 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:04.049685 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0392b48d-da41-4672-9fe6-86b91db89320 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:04.068733 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 920/1812] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:30:04.073744 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d17f221-ebfd-4b5f-b053-9ebbbda10734 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:04.081540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0392b48d-da41-4672-9fe6-86b91db89320 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:04.089080 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543bc7d7-4a56-4b4a-a18e-e6cf12ee42c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:04.094527 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543bc7d7-4a56-4b4a-a18e-e6cf12ee42c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:04.095880 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543bc7d7-4a56-4b4a-a18e-e6cf12ee42c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:04.098138 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543bc7d7-4a56-4b4a-a18e-e6cf12ee42c1 admin admin] PATH_INFO: `/v3/projects/92e13ef51b494757aab7bad432e979a2` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:04.101685 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543bc7d7-4a56-4b4a-a18e-e6cf12ee42c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92e13ef51b494757aab7bad432e979a2)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:04.107491 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543bc7d7-4a56-4b4a-a18e-e6cf12ee42c1 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:04.133562 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d17f221-ebfd-4b5f-b053-9ebbbda10734 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:04.157729 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6126ec-6da1-4ba1-8bc5-440f48c121ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:04.159363 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6126ec-6da1-4ba1-8bc5-440f48c121ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:04.164063 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6126ec-6da1-4ba1-8bc5-440f48c121ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:04.164982 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6126ec-6da1-4ba1-8bc5-440f48c121ab admin admin] PATH_INFO: `/v3/projects/ff04efdb5c144487ac8258f1cbf4c174/users/ed2ea8a03815421791c6e57c6d58e782/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:04.170100 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6126ec-6da1-4ba1-8bc5-440f48c121ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff04efdb5c144487ac8258f1cbf4c174, user_id=ed2ea8a03815421791c6e57c6d58e782, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:04.176618 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6126ec-6da1-4ba1-8bc5-440f48c121ab admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:04.213672 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 921/1813] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:30:04 2022] PUT /identity/v3/projects/ff04efdb5c144487ac8258f1cbf4c174/users/ed2ea8a03815421791c6e57c6d58e782/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:30:04.229778 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c460598e-f5f3-4404-976b-a474299003c9 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:04.230769 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c460598e-f5f3-4404-976b-a474299003c9 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:04.231496 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c460598e-f5f3-4404-976b-a474299003c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:04.282355 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-543bc7d7-4a56-4b4a-a18e-e6cf12ee42c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92e13ef51b494757aab7bad432e979a2', 'request_id': 'req-543bc7d7-4a56-4b4a-a18e-e6cf12ee42c1'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:04.312798 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-543bc7d7-4a56-4b4a-a18e-e6cf12ee42c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92e13ef51b494757aab7bad432e979a2', 'request_id': 'req-543bc7d7-4a56-4b4a-a18e-e6cf12ee42c1'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:04.322864 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-c460598e-f5f3-4404-976b-a474299003c9 None None] MFA Rules not processed for user `ed2ea8a03815421791c6e57c6d58e782`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:04.325178 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c460598e-f5f3-4404-976b-a474299003c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:04.333445 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-543bc7d7-4a56-4b4a-a18e-e6cf12ee42c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92e13ef51b494757aab7bad432e979a2', 'request_id': 'req-543bc7d7-4a56-4b4a-a18e-e6cf12ee42c1'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:04.364107 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:04.389430 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-543bc7d7-4a56-4b4a-a18e-e6cf12ee42c1 admin admin] The token cache is being invalidate because project 92e13ef51b494757aab7bad432e979a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:30:04.389813 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-543bc7d7-4a56-4b4a-a18e-e6cf12ee42c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:04.392209 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-543bc7d7-4a56-4b4a-a18e-e6cf12ee42c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:04.400923 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 893/1814] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:30:04 2022] DELETE /identity/v3/projects/92e13ef51b494757aab7bad432e979a2 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:04.408828 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b633d30-d988-4ee2-892c-7dceb774ee6b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:04.410054 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b633d30-d988-4ee2-892c-7dceb774ee6b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:04.410441 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b633d30-d988-4ee2-892c-7dceb774ee6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:04.450349 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-4b633d30-d988-4ee2-892c-7dceb774ee6b None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:04.451743 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4b633d30-d988-4ee2-892c-7dceb774ee6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:04.530358 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:04.553372 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 922/1815] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:04 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:04.572607 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c460598e-f5f3-4404-976b-a474299003c9 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:04.575102 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c460598e-f5f3-4404-976b-a474299003c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:04.596797 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 894/1816] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:04.616471 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac24c2e-9514-4b0b-94db-7d97e4b79153 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:04.617548 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac24c2e-9514-4b0b-94db-7d97e4b79153 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:04.618198 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac24c2e-9514-4b0b-94db-7d97e4b79153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:04.655842 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-9ac24c2e-9514-4b0b-94db-7d97e4b79153 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:04.658719 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9ac24c2e-9514-4b0b-94db-7d97e4b79153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:04.677286 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 895/1817] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:04.679300 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac24c2e-9514-4b0b-94db-7d97e4b79153 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:04.697280 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac24c2e-9514-4b0b-94db-7d97e4b79153 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:04.705616 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0f8927-1896-4b55-b75a-a57289508728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:04.706762 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f8927-1896-4b55-b75a-a57289508728 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:04.707208 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f8927-1896-4b55-b75a-a57289508728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:04.707655 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f8927-1896-4b55-b75a-a57289508728 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:04.708972 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0f8927-1896-4b55-b75a-a57289508728 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:30:04.709509 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0f8927-1896-4b55-b75a-a57289508728 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:04.712639 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0f8927-1896-4b55-b75a-a57289508728 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:04.714929 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c460598e-f5f3-4404-976b-a474299003c9 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:04.734766 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2756be-3c9a-47c4-a232-99abcc2857db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:04.736448 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2756be-3c9a-47c4-a232-99abcc2857db service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:04.737404 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2756be-3c9a-47c4-a232-99abcc2857db service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:04.737939 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2756be-3c9a-47c4-a232-99abcc2857db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:04.748419 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 896/1818] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:30:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:30:04.749467 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0f8927-1896-4b55-b75a-a57289508728 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:04.757143 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2756be-3c9a-47c4-a232-99abcc2857db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:04.759937 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2756be-3c9a-47c4-a232-99abcc2857db service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:04.774835 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0f8927-1896-4b55-b75a-a57289508728 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:04.779124 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 923/1819] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:30:04 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:30:04.782959 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c95775-3921-4315-bfe1-a1ed62d6b32b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:04.783831 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c95775-3921-4315-bfe1-a1ed62d6b32b admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:04.784249 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c95775-3921-4315-bfe1-a1ed62d6b32b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:04.784686 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c95775-3921-4315-bfe1-a1ed62d6b32b admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:04.785930 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c95775-3921-4315-bfe1-a1ed62d6b32b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:04.789474 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c95775-3921-4315-bfe1-a1ed62d6b32b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:04.876077 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 897/1820] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:30:04 2022] POST /identity/v3/projects => generated 367 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:30:04.878125 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2756be-3c9a-47c4-a232-99abcc2857db service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:04.905619 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2756be-3c9a-47c4-a232-99abcc2857db service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:05.078153 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02b5471-b5ea-4230-bdfe-06326976aa44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:05.080292 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b5471-b5ea-4230-bdfe-06326976aa44 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:05.080708 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b5471-b5ea-4230-bdfe-06326976aa44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:05.081194 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b5471-b5ea-4230-bdfe-06326976aa44 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:05.082537 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02b5471-b5ea-4230-bdfe-06326976aa44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:05.086122 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02b5471-b5ea-4230-bdfe-06326976aa44 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:05.124024 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 924/1821] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:30:04 2022] POST /identity/v3/users => generated 284 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:30:05.125980 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c95775-3921-4315-bfe1-a1ed62d6b32b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:05.139784 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c95775-3921-4315-bfe1-a1ed62d6b32b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:05.148328 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ea474b-28aa-437c-bce5-132a0c7a1c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:05.149791 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ea474b-28aa-437c-bce5-132a0c7a1c5e admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:05.150280 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ea474b-28aa-437c-bce5-132a0c7a1c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:05.150736 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ea474b-28aa-437c-bce5-132a0c7a1c5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:05.151755 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ea474b-28aa-437c-bce5-132a0c7a1c5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:05.153739 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ea474b-28aa-437c-bce5-132a0c7a1c5e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:05.176412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 898/1822] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:30:05.181774 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02b5471-b5ea-4230-bdfe-06326976aa44 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:05.199211 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02b5471-b5ea-4230-bdfe-06326976aa44 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:05.217273 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f8f0c1-3cb2-4512-9963-2b52c29977f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:05.218558 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8f0c1-3cb2-4512-9963-2b52c29977f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:05.219677 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8f0c1-3cb2-4512-9963-2b52c29977f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:05.220122 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8f0c1-3cb2-4512-9963-2b52c29977f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:05.221297 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f8f0c1-3cb2-4512-9963-2b52c29977f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:05.224134 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f8f0c1-3cb2-4512-9963-2b52c29977f3 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:05.241252 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 925/1823] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:30:05.245970 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ea474b-28aa-437c-bce5-132a0c7a1c5e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:05.270899 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ea474b-28aa-437c-bce5-132a0c7a1c5e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:05.277513 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea0477c-f418-43fe-90ed-29f649c2e203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:05.278559 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0477c-f418-43fe-90ed-29f649c2e203 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:05.278908 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0477c-f418-43fe-90ed-29f649c2e203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:05.279608 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0477c-f418-43fe-90ed-29f649c2e203 admin admin] PATH_INFO: `/v3/projects/a6ab8d0c96c44844a463d38d9b001032/users/9616e0d220134980aa955dd453af36a4/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:05.330171 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea0477c-f418-43fe-90ed-29f649c2e203 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6ab8d0c96c44844a463d38d9b001032, user_id=9616e0d220134980aa955dd453af36a4, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:05.337573 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea0477c-f418-43fe-90ed-29f649c2e203 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:05.372353 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 899/1824] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:30:05 2022] PUT /identity/v3/projects/a6ab8d0c96c44844a463d38d9b001032/users/9616e0d220134980aa955dd453af36a4/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:05.394966 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190cca51-fa11-4ea1-ae6b-bee40acc01a4 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:05.395710 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190cca51-fa11-4ea1-ae6b-bee40acc01a4 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:05.396421 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190cca51-fa11-4ea1-ae6b-bee40acc01a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:05.450618 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-190cca51-fa11-4ea1-ae6b-bee40acc01a4 None None] MFA Rules not processed for user `9616e0d220134980aa955dd453af36a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:05.452619 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-190cca51-fa11-4ea1-ae6b-bee40acc01a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:05.498654 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:05.681092 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 926/1825] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:05 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:05.688916 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea0477c-f418-43fe-90ed-29f649c2e203 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:05.792325 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190cca51-fa11-4ea1-ae6b-bee40acc01a4 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:05.795482 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-190cca51-fa11-4ea1-ae6b-bee40acc01a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:05.966251 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea0477c-f418-43fe-90ed-29f649c2e203 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:05.988965 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c385054-3ca9-493a-8518-36e0b81654d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:05.995150 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c385054-3ca9-493a-8518-36e0b81654d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:05.995898 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c385054-3ca9-493a-8518-36e0b81654d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:05.999581 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c385054-3ca9-493a-8518-36e0b81654d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:06.003096 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c385054-3ca9-493a-8518-36e0b81654d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:06.010581 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c385054-3ca9-493a-8518-36e0b81654d8 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:06.056943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190cca51-fa11-4ea1-ae6b-bee40acc01a4 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:06.070742 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf21a17-dc3a-4d27-a41d-8ab41c4ff07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:06.072655 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf21a17-dc3a-4d27-a41d-8ab41c4ff07c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:06.072655 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf21a17-dc3a-4d27-a41d-8ab41c4ff07c admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:06.072655 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf21a17-dc3a-4d27-a41d-8ab41c4ff07c admin admin] PATH_INFO: `/v3/users/d93c2ac7de994271843cfacca9a4d9ec` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:06.077318 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf21a17-dc3a-4d27-a41d-8ab41c4ff07c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d93c2ac7de994271843cfacca9a4d9ec)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:06.080069 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf21a17-dc3a-4d27-a41d-8ab41c4ff07c admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:06.105513 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 900/1826] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:30:05 2022] POST /identity/v3/projects => generated 367 bytes in 417 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:30:06.112787 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c385054-3ca9-493a-8518-36e0b81654d8 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:06.192115 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-fbf21a17-dc3a-4d27-a41d-8ab41c4ff07c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd93c2ac7de994271843cfacca9a4d9ec'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:06.253750 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-fbf21a17-dc3a-4d27-a41d-8ab41c4ff07c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd93c2ac7de994271843cfacca9a4d9ec'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:06.256500 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-fbf21a17-dc3a-4d27-a41d-8ab41c4ff07c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd93c2ac7de994271843cfacca9a4d9ec'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:06.260515 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-fbf21a17-dc3a-4d27-a41d-8ab41c4ff07c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd93c2ac7de994271843cfacca9a4d9ec'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:06.277118 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-fbf21a17-dc3a-4d27-a41d-8ab41c4ff07c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd93c2ac7de994271843cfacca9a4d9ec'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:06.301038 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:06.301781 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 927/1827] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:30:05 2022] DELETE /identity/v3/users/d93c2ac7de994271843cfacca9a4d9ec => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:06.305310 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d811a-2573-4f86-8f98-30ab1c8cd276 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:06.305737 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d811a-2573-4f86-8f98-30ab1c8cd276 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:06.305971 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d811a-2573-4f86-8f98-30ab1c8cd276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:06.333721 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-a46d811a-2573-4f86-8f98-30ab1c8cd276 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:06.336200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a46d811a-2573-4f86-8f98-30ab1c8cd276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:06.369045 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c385054-3ca9-493a-8518-36e0b81654d8 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:06.402506 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:06.487470 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 928/1828] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:06.501847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46d811a-2573-4f86-8f98-30ab1c8cd276 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:06.504016 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a46d811a-2573-4f86-8f98-30ab1c8cd276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:06.577692 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b290fb91-7518-429d-9ef8-237ee5c58f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:06.582419 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290fb91-7518-429d-9ef8-237ee5c58f7f admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:06.584693 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290fb91-7518-429d-9ef8-237ee5c58f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:06.585237 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290fb91-7518-429d-9ef8-237ee5c58f7f admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:06.591507 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b290fb91-7518-429d-9ef8-237ee5c58f7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:06.599514 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b290fb91-7518-429d-9ef8-237ee5c58f7f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:06.668662 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 901/1829] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:30:06 2022] POST /identity/v3/users => generated 290 bytes in 558 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:30:06.675349 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b290fb91-7518-429d-9ef8-237ee5c58f7f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:06.678187 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b290fb91-7518-429d-9ef8-237ee5c58f7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:06.717658 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46d811a-2573-4f86-8f98-30ab1c8cd276 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:06.730387 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab7b93d-8b32-4cab-bdaa-9b4d85ee62c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:06.731134 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab7b93d-8b32-4cab-bdaa-9b4d85ee62c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:06.731430 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab7b93d-8b32-4cab-bdaa-9b4d85ee62c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:06.731713 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab7b93d-8b32-4cab-bdaa-9b4d85ee62c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:06.760380 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b290fb91-7518-429d-9ef8-237ee5c58f7f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:06.770443 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab7b93d-8b32-4cab-bdaa-9b4d85ee62c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:06.776618 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab7b93d-8b32-4cab-bdaa-9b4d85ee62c9 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:06.791396 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671b6699-4669-4b87-94bc-6ada473ff952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:06.794493 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b6699-4669-4b87-94bc-6ada473ff952 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:06.801120 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b6699-4669-4b87-94bc-6ada473ff952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:06.802002 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b6699-4669-4b87-94bc-6ada473ff952 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:06.806900 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671b6699-4669-4b87-94bc-6ada473ff952 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:06.818070 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671b6699-4669-4b87-94bc-6ada473ff952 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:06.831911 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 929/1830] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:30:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:30:06.871499 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 902/1831] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:30:06.873307 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab7b93d-8b32-4cab-bdaa-9b4d85ee62c9 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:06.901967 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab7b93d-8b32-4cab-bdaa-9b4d85ee62c9 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:06.926922 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afad7f52-6194-4bd5-a149-748ff88fa194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:06.928413 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad7f52-6194-4bd5-a149-748ff88fa194 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:06.928887 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad7f52-6194-4bd5-a149-748ff88fa194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:06.929341 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad7f52-6194-4bd5-a149-748ff88fa194 admin admin] PATH_INFO: `/v3/projects/4c877ece2ebe43138b7b83661eee5392/users/66d9f6e69fa746489b5821b5a7a7eceb/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:06.973128 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afad7f52-6194-4bd5-a149-748ff88fa194 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c877ece2ebe43138b7b83661eee5392, user_id=66d9f6e69fa746489b5821b5a7a7eceb, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:06.994107 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afad7f52-6194-4bd5-a149-748ff88fa194 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:07.032746 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 930/1832] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:30:06 2022] PUT /identity/v3/projects/4c877ece2ebe43138b7b83661eee5392/users/66d9f6e69fa746489b5821b5a7a7eceb/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:30:07.034074 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671b6699-4669-4b87-94bc-6ada473ff952 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:07.081880 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671b6699-4669-4b87-94bc-6ada473ff952 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:07.555230 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974590ad-ec6e-4ed7-8233-30c991f273c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:07.558766 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974590ad-ec6e-4ed7-8233-30c991f273c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:07.565970 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974590ad-ec6e-4ed7-8233-30c991f273c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:07.565970 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974590ad-ec6e-4ed7-8233-30c991f273c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:07.569097 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974590ad-ec6e-4ed7-8233-30c991f273c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:07.582478 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974590ad-ec6e-4ed7-8233-30c991f273c0 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:07.592409 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ce16a-3820-4a20-aa5d-c26ca3e5adf4 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:07.593413 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ce16a-3820-4a20-aa5d-c26ca3e5adf4 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:07.594004 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ce16a-3820-4a20-aa5d-c26ca3e5adf4 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:07.598544 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 931/1833] 10.222.0.87 () {54 vars in 921 bytes} [Tue Jun 21 18:30:07 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:30:07.638949 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cce58f-7797-4928-90d7-eba29a70a3a1 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:07.639300 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cce58f-7797-4928-90d7-eba29a70a3a1 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:07.639637 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cce58f-7797-4928-90d7-eba29a70a3a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:07.663776 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 903/1834] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:07 2022] GET /identity/v3/roles => generated 1420 bytes in 629 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:30:07.668658 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974590ad-ec6e-4ed7-8233-30c991f273c0 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:07.691313 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-17cce58f-7797-4928-90d7-eba29a70a3a1 None None] MFA Rules not processed for user `a1b3fa67905c4027b3656ac36e2b76c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:07.696232 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-17cce58f-7797-4928-90d7-eba29a70a3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:07.722882 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974590ad-ec6e-4ed7-8233-30c991f273c0 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:07.758818 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d4c005-f348-4ae2-8863-78ba118c3d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:07.766607 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d4c005-f348-4ae2-8863-78ba118c3d15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:07.769057 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d4c005-f348-4ae2-8863-78ba118c3d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:07.776924 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d4c005-f348-4ae2-8863-78ba118c3d15 admin admin] PATH_INFO: `/v3/projects/4c877ece2ebe43138b7b83661eee5392/users/66d9f6e69fa746489b5821b5a7a7eceb/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:07.789606 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d4c005-f348-4ae2-8863-78ba118c3d15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c877ece2ebe43138b7b83661eee5392, user_id=66d9f6e69fa746489b5821b5a7a7eceb, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:07.808505 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d4c005-f348-4ae2-8863-78ba118c3d15 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:07.865420 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 904/1835] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:30:07 2022] PUT /identity/v3/projects/4c877ece2ebe43138b7b83661eee5392/users/66d9f6e69fa746489b5821b5a7a7eceb/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:07.895044 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f9e2c7-f66f-4091-961a-62ffa93ab56a None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:07.895465 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f9e2c7-f66f-4091-961a-62ffa93ab56a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:07.895796 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f9e2c7-f66f-4091-961a-62ffa93ab56a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:07.938876 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 932/1836] 10.222.0.87 () {58 vars in 1018 bytes} [Tue Jun 21 18:30:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:30:07.963552 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cce58f-7797-4928-90d7-eba29a70a3a1 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:07.966526 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-17cce58f-7797-4928-90d7-eba29a70a3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:08.004132 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-45f9e2c7-f66f-4091-961a-62ffa93ab56a None None] MFA Rules not processed for user `66d9f6e69fa746489b5821b5a7a7eceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:08.006710 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-45f9e2c7-f66f-4091-961a-62ffa93ab56a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:08.104831 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cce58f-7797-4928-90d7-eba29a70a3a1 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:08.145544 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:08.272008 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 905/1837] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:07 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:30:08.285146 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f9e2c7-f66f-4091-961a-62ffa93ab56a None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:08.287940 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-45f9e2c7-f66f-4091-961a-62ffa93ab56a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:08.337465 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:08.450459 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d98878-b3e0-4e4b-9e45-d7a09204f179 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:08.451127 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d98878-b3e0-4e4b-9e45-d7a09204f179 service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:08.451631 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d98878-b3e0-4e4b-9e45-d7a09204f179 service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:08.452344 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d98878-b3e0-4e4b-9e45-d7a09204f179 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:08.460698 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f9e2c7-f66f-4091-961a-62ffa93ab56a None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:08.474189 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd8880-877b-4b7c-851c-5e031327ba76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:08.475303 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd8880-877b-4b7c-851c-5e031327ba76 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:08.476266 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd8880-877b-4b7c-851c-5e031327ba76 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:08.476382 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd8880-877b-4b7c-851c-5e031327ba76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:08.494449 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d98878-b3e0-4e4b-9e45-d7a09204f179 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:08.496793 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d98878-b3e0-4e4b-9e45-d7a09204f179 service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:08.504270 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fd8880-877b-4b7c-851c-5e031327ba76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:08.507199 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fd8880-877b-4b7c-851c-5e031327ba76 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:08.532944 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 933/1838] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:30:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:30:08.533999 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d98878-b3e0-4e4b-9e45-d7a09204f179 service nova] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:08.536345 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 906/1839] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:30:08 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:30:08.549054 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-08d98878-b3e0-4e4b-9e45-d7a09204f179 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:08.903042 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd8880-877b-4b7c-851c-5e031327ba76 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:08.907883 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-00fd8880-877b-4b7c-851c-5e031327ba76 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:09.028857 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd8880-877b-4b7c-851c-5e031327ba76 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:09.072149 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c3b907-71ff-4aa8-a574-6c1271b3390b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:09.073843 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3b907-71ff-4aa8-a574-6c1271b3390b service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:09.076818 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3b907-71ff-4aa8-a574-6c1271b3390b service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:09.081965 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3b907-71ff-4aa8-a574-6c1271b3390b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:09.091296 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a8c3b907-71ff-4aa8-a574-6c1271b3390b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:09.125053 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d98878-b3e0-4e4b-9e45-d7a09204f179 service nova] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:09.158581 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb76596e-b1b4-44cc-b39e-5f287f481bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:09.164068 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb76596e-b1b4-44cc-b39e-5f287f481bfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:09.169813 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb76596e-b1b4-44cc-b39e-5f287f481bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:09.170358 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb76596e-b1b4-44cc-b39e-5f287f481bfe admin admin] PATH_INFO: `/v3/projects/a1a6aa18cdc9405895e8a9072bc60ec4` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:09.177281 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb76596e-b1b4-44cc-b39e-5f287f481bfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1a6aa18cdc9405895e8a9072bc60ec4)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:09.189457 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb76596e-b1b4-44cc-b39e-5f287f481bfe admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:09.390481 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c3b907-71ff-4aa8-a574-6c1271b3390b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:09.399018 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c3b907-71ff-4aa8-a574-6c1271b3390b service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:09.436400 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-eb76596e-b1b4-44cc-b39e-5f287f481bfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1a6aa18cdc9405895e8a9072bc60ec4', 'request_id': 'req-eb76596e-b1b4-44cc-b39e-5f287f481bfe'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:09.456728 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 907/1840] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:30:08 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:30:09.464305 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-eb76596e-b1b4-44cc-b39e-5f287f481bfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1a6aa18cdc9405895e8a9072bc60ec4', 'request_id': 'req-eb76596e-b1b4-44cc-b39e-5f287f481bfe'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:09.500423 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-eb76596e-b1b4-44cc-b39e-5f287f481bfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1a6aa18cdc9405895e8a9072bc60ec4', 'request_id': 'req-eb76596e-b1b4-44cc-b39e-5f287f481bfe'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:09.560739 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-eb76596e-b1b4-44cc-b39e-5f287f481bfe admin admin] The token cache is being invalidate because project a1a6aa18cdc9405895e8a9072bc60ec4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:30:09.561208 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-eb76596e-b1b4-44cc-b39e-5f287f481bfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:09.565711 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-eb76596e-b1b4-44cc-b39e-5f287f481bfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:09.572004 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 934/1841] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:30:08 2022] DELETE /identity/v3/projects/a1a6aa18cdc9405895e8a9072bc60ec4 => generated 0 bytes in 1038 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:09.577970 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c3b907-71ff-4aa8-a574-6c1271b3390b service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:09.599496 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c3b907-71ff-4aa8-a574-6c1271b3390b service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:09.847077 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9547af83-dfd5-4197-95de-8aab1fa01384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:09.848790 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9547af83-dfd5-4197-95de-8aab1fa01384 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:09.849225 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9547af83-dfd5-4197-95de-8aab1fa01384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:09.849607 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9547af83-dfd5-4197-95de-8aab1fa01384 admin admin] PATH_INFO: `/v3/users/ed2ea8a03815421791c6e57c6d58e782` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:09.851979 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9547af83-dfd5-4197-95de-8aab1fa01384 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed2ea8a03815421791c6e57c6d58e782)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:09.854735 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9547af83-dfd5-4197-95de-8aab1fa01384 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:09.928870 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:10.014209 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9547af83-dfd5-4197-95de-8aab1fa01384 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed2ea8a03815421791c6e57c6d58e782'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:10.040034 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9547af83-dfd5-4197-95de-8aab1fa01384 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed2ea8a03815421791c6e57c6d58e782'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:10.045311 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9547af83-dfd5-4197-95de-8aab1fa01384 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed2ea8a03815421791c6e57c6d58e782'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:10.047105 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9547af83-dfd5-4197-95de-8aab1fa01384 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed2ea8a03815421791c6e57c6d58e782'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:10.154470 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9547af83-dfd5-4197-95de-8aab1fa01384 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed2ea8a03815421791c6e57c6d58e782'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:10.192707 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 908/1842] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:30:09 2022] DELETE /identity/v3/users/ed2ea8a03815421791c6e57c6d58e782 => generated 0 bytes in 615 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:11.156090 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb76596e-b1b4-44cc-b39e-5f287f481bfe admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:11.158187 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-eb76596e-b1b4-44cc-b39e-5f287f481bfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:11.174280 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb76596e-b1b4-44cc-b39e-5f287f481bfe admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:11.310874 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c0a5d-da50-4f4c-a3a4-e4b1b16e7df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:11.311782 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c0a5d-da50-4f4c-a3a4-e4b1b16e7df4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:11.312234 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c0a5d-da50-4f4c-a3a4-e4b1b16e7df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:11.312788 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c0a5d-da50-4f4c-a3a4-e4b1b16e7df4 admin admin] PATH_INFO: `/v3/projects/ff04efdb5c144487ac8258f1cbf4c174` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:11.314153 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91c0a5d-da50-4f4c-a3a4-e4b1b16e7df4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff04efdb5c144487ac8258f1cbf4c174)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:11.317321 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91c0a5d-da50-4f4c-a3a4-e4b1b16e7df4 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:11.321535 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:11.475797 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e91c0a5d-da50-4f4c-a3a4-e4b1b16e7df4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff04efdb5c144487ac8258f1cbf4c174', 'request_id': 'req-e91c0a5d-da50-4f4c-a3a4-e4b1b16e7df4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:11.486458 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e91c0a5d-da50-4f4c-a3a4-e4b1b16e7df4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff04efdb5c144487ac8258f1cbf4c174', 'request_id': 'req-e91c0a5d-da50-4f4c-a3a4-e4b1b16e7df4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:11.512720 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e91c0a5d-da50-4f4c-a3a4-e4b1b16e7df4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff04efdb5c144487ac8258f1cbf4c174', 'request_id': 'req-e91c0a5d-da50-4f4c-a3a4-e4b1b16e7df4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:11.535888 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e91c0a5d-da50-4f4c-a3a4-e4b1b16e7df4 admin admin] The token cache is being invalidate because project ff04efdb5c144487ac8258f1cbf4c174 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:30:11.537795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e91c0a5d-da50-4f4c-a3a4-e4b1b16e7df4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:11.539653 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e91c0a5d-da50-4f4c-a3a4-e4b1b16e7df4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:11.551717 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 935/1843] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:30:11 2022] DELETE /identity/v3/projects/ff04efdb5c144487ac8258f1cbf4c174 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:30:11.555828 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944db2b1-bf76-4aae-9b85-b6ff536ed1e7 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:11.557025 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944db2b1-bf76-4aae-9b85-b6ff536ed1e7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:11.557996 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944db2b1-bf76-4aae-9b85-b6ff536ed1e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:11.602730 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-944db2b1-bf76-4aae-9b85-b6ff536ed1e7 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:11.622188 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-944db2b1-bf76-4aae-9b85-b6ff536ed1e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:11.774073 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 909/1844] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:11.786959 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4931a-db1f-4727-bbd1-d2b65cae9423 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:11.787270 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4931a-db1f-4727-bbd1-d2b65cae9423 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:11.787708 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4931a-db1f-4727-bbd1-d2b65cae9423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:11.832531 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-31f4931a-db1f-4727-bbd1-d2b65cae9423 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:11.835260 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-31f4931a-db1f-4727-bbd1-d2b65cae9423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:11.864967 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 936/1845] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:11.877569 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944db2b1-bf76-4aae-9b85-b6ff536ed1e7 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:11.901395 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944db2b1-bf76-4aae-9b85-b6ff536ed1e7 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:11.934822 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f0b03-c15b-49be-ae3c-186a53552037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:11.936619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f0b03-c15b-49be-ae3c-186a53552037 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:11.939730 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f0b03-c15b-49be-ae3c-186a53552037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:11.941586 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f0b03-c15b-49be-ae3c-186a53552037 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:11.942913 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447f0b03-c15b-49be-ae3c-186a53552037 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:30:11.947931 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447f0b03-c15b-49be-ae3c-186a53552037 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:11.950661 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447f0b03-c15b-49be-ae3c-186a53552037 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:12.001477 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 910/1846] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:30:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:30:12.003918 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f4931a-db1f-4727-bbd1-d2b65cae9423 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:12.023030 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f4931a-db1f-4727-bbd1-d2b65cae9423 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:12.032487 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73cc50-6452-4944-880c-11c3401f5ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:12.033219 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73cc50-6452-4944-880c-11c3401f5ffc admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:12.033511 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73cc50-6452-4944-880c-11c3401f5ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:12.033824 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73cc50-6452-4944-880c-11c3401f5ffc admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:12.035221 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e73cc50-6452-4944-880c-11c3401f5ffc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:12.037301 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e73cc50-6452-4944-880c-11c3401f5ffc admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:12.126953 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 937/1847] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:30:12 2022] POST /identity/v3/projects => generated 383 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:30:12.127399 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f0b03-c15b-49be-ae3c-186a53552037 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:12.153326 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f0b03-c15b-49be-ae3c-186a53552037 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:12.279889 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1388db-0571-4012-90d3-a27588494060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:12.281455 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1388db-0571-4012-90d3-a27588494060 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:12.281876 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1388db-0571-4012-90d3-a27588494060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:12.283620 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1388db-0571-4012-90d3-a27588494060 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:12.284583 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1388db-0571-4012-90d3-a27588494060 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:12.290961 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1388db-0571-4012-90d3-a27588494060 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:12.365690 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 911/1848] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:30:12 2022] POST /identity/v3/users => generated 292 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:30:12.369873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73cc50-6452-4944-880c-11c3401f5ffc admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:12.395469 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73cc50-6452-4944-880c-11c3401f5ffc admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:12.407949 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f591d-0d78-4204-8413-40d28b4f0bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:12.409618 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f591d-0d78-4204-8413-40d28b4f0bc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:12.410661 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f591d-0d78-4204-8413-40d28b4f0bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:12.411269 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f591d-0d78-4204-8413-40d28b4f0bc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:12.412807 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f591d-0d78-4204-8413-40d28b4f0bc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:12.419027 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f591d-0d78-4204-8413-40d28b4f0bc1 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:12.445723 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 938/1849] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:12 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:30:12.450273 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1388db-0571-4012-90d3-a27588494060 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:12.472847 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1388db-0571-4012-90d3-a27588494060 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:12.480114 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fd11df-2408-4d98-9099-da7a652b6a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:12.481135 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd11df-2408-4d98-9099-da7a652b6a1d admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:12.483224 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd11df-2408-4d98-9099-da7a652b6a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:12.483916 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd11df-2408-4d98-9099-da7a652b6a1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:12.485512 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fd11df-2408-4d98-9099-da7a652b6a1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:12.490267 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fd11df-2408-4d98-9099-da7a652b6a1d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:12.511773 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 912/1850] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:12 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:30:12.516851 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f591d-0d78-4204-8413-40d28b4f0bc1 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:12.539865 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f591d-0d78-4204-8413-40d28b4f0bc1 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:12.539865 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c47cc5f-b766-4d98-ab24-a3ed88b9d903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:12.540611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47cc5f-b766-4d98-ab24-a3ed88b9d903 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:12.540788 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47cc5f-b766-4d98-ab24-a3ed88b9d903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:12.541730 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47cc5f-b766-4d98-ab24-a3ed88b9d903 admin admin] PATH_INFO: `/v3/projects/9dc4b89703074c6f9709c99a45a94f29/users/cc4aca40ca4e4d48be6830e5ea067246/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:12.564826 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c47cc5f-b766-4d98-ab24-a3ed88b9d903 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dc4b89703074c6f9709c99a45a94f29, user_id=cc4aca40ca4e4d48be6830e5ea067246, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:12.568665 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c47cc5f-b766-4d98-ab24-a3ed88b9d903 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:12.610506 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 939/1851] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:30:12 2022] PUT /identity/v3/projects/9dc4b89703074c6f9709c99a45a94f29/users/cc4aca40ca4e4d48be6830e5ea067246/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:30:12.636683 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d16d1b-f7f8-416f-9488-2820c5956d7a None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:12.637391 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d16d1b-f7f8-416f-9488-2820c5956d7a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:12.637391 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d16d1b-f7f8-416f-9488-2820c5956d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:12.654432 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:12.732883 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f2d16d1b-f7f8-416f-9488-2820c5956d7a None None] MFA Rules not processed for user `cc4aca40ca4e4d48be6830e5ea067246`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:12.739797 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f2d16d1b-f7f8-416f-9488-2820c5956d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:12.939311 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 913/1852] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:30:12.944449 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c47cc5f-b766-4d98-ab24-a3ed88b9d903 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:13.121178 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c47cc5f-b766-4d98-ab24-a3ed88b9d903 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:13.138552 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833cde39-b3e7-408e-9510-43fa609bdbe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:13.139368 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833cde39-b3e7-408e-9510-43fa609bdbe7 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:13.139655 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833cde39-b3e7-408e-9510-43fa609bdbe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:13.139946 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833cde39-b3e7-408e-9510-43fa609bdbe7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:13.140746 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833cde39-b3e7-408e-9510-43fa609bdbe7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:13.142965 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833cde39-b3e7-408e-9510-43fa609bdbe7 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:13.191004 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 940/1853] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:30:12 2022] POST /identity/v3/projects => generated 383 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:30:13.197890 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d16d1b-f7f8-416f-9488-2820c5956d7a None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:13.237078 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d16d1b-f7f8-416f-9488-2820c5956d7a None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:13.402315 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28e596f-dcbe-4e29-a811-e2cd6b57d742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:13.403154 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e596f-dcbe-4e29-a811-e2cd6b57d742 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:13.403513 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e596f-dcbe-4e29-a811-e2cd6b57d742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:13.404016 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e596f-dcbe-4e29-a811-e2cd6b57d742 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:13.404836 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28e596f-dcbe-4e29-a811-e2cd6b57d742 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:13.407042 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28e596f-dcbe-4e29-a811-e2cd6b57d742 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:13.450361 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 914/1854] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:30:13 2022] POST /identity/v3/users => generated 298 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:30:13.454808 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833cde39-b3e7-408e-9510-43fa609bdbe7 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:13.468018 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833cde39-b3e7-408e-9510-43fa609bdbe7 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:13.474296 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e551f-4d9f-4c08-848e-c80e1dafbcaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:13.475179 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e551f-4d9f-4c08-848e-c80e1dafbcaf admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:13.475470 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e551f-4d9f-4c08-848e-c80e1dafbcaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:13.475755 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e551f-4d9f-4c08-848e-c80e1dafbcaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:13.476505 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e551f-4d9f-4c08-848e-c80e1dafbcaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:13.478234 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e551f-4d9f-4c08-848e-c80e1dafbcaf admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:13.495607 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 941/1855] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:13 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:30:13.497545 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28e596f-dcbe-4e29-a811-e2cd6b57d742 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:13.562600 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28e596f-dcbe-4e29-a811-e2cd6b57d742 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:13.583318 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27225ffa-0053-478d-9aaa-0be75429bc0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:13.584836 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27225ffa-0053-478d-9aaa-0be75429bc0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:13.585315 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27225ffa-0053-478d-9aaa-0be75429bc0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:13.590183 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27225ffa-0053-478d-9aaa-0be75429bc0c admin admin] PATH_INFO: `/v3/projects/0211aa0e968c475a936d67fe00052b21/users/92454b671c434427857c73e135f2fd09/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:13.617081 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27225ffa-0053-478d-9aaa-0be75429bc0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0211aa0e968c475a936d67fe00052b21, user_id=92454b671c434427857c73e135f2fd09, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:13.620385 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27225ffa-0053-478d-9aaa-0be75429bc0c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:13.650669 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e551f-4d9f-4c08-848e-c80e1dafbcaf admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:13.651010 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 915/1856] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:30:13 2022] PUT /identity/v3/projects/0211aa0e968c475a936d67fe00052b21/users/92454b671c434427857c73e135f2fd09/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:13.669617 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:13.687274 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e551f-4d9f-4c08-848e-c80e1dafbcaf admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:13.826674 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204135fc-715f-48d1-b348-d6cec5c57e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:13.828087 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204135fc-715f-48d1-b348-d6cec5c57e09 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:13.828087 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204135fc-715f-48d1-b348-d6cec5c57e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:13.828415 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204135fc-715f-48d1-b348-d6cec5c57e09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:13.829877 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204135fc-715f-48d1-b348-d6cec5c57e09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:13.831700 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204135fc-715f-48d1-b348-d6cec5c57e09 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:13.870674 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 942/1857] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:13 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:30:13.873151 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27225ffa-0053-478d-9aaa-0be75429bc0c admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:13.909343 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27225ffa-0053-478d-9aaa-0be75429bc0c admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:13.918022 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3bb35a-73c8-41c7-9a4c-33292a599224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:13.919600 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3bb35a-73c8-41c7-9a4c-33292a599224 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:13.925895 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3bb35a-73c8-41c7-9a4c-33292a599224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:13.925895 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3bb35a-73c8-41c7-9a4c-33292a599224 admin admin] PATH_INFO: `/v3/projects/0211aa0e968c475a936d67fe00052b21/users/92454b671c434427857c73e135f2fd09/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:13.929678 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3bb35a-73c8-41c7-9a4c-33292a599224 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0211aa0e968c475a936d67fe00052b21, user_id=92454b671c434427857c73e135f2fd09, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:13.934920 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3bb35a-73c8-41c7-9a4c-33292a599224 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:13.967664 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 916/1858] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:30:13 2022] PUT /identity/v3/projects/0211aa0e968c475a936d67fe00052b21/users/92454b671c434427857c73e135f2fd09/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:13.982381 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a6e6d-d9f5-4c6c-af3a-bcd03c84e0fc None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:13.984954 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a6e6d-d9f5-4c6c-af3a-bcd03c84e0fc None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:13.986183 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a6e6d-d9f5-4c6c-af3a-bcd03c84e0fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:14.063099 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-637a6e6d-d9f5-4c6c-af3a-bcd03c84e0fc None None] MFA Rules not processed for user `92454b671c434427857c73e135f2fd09`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:14.065555 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-637a6e6d-d9f5-4c6c-af3a-bcd03c84e0fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:14.270128 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 943/1859] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:13 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:14.301291 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3bb35a-73c8-41c7-9a4c-33292a599224 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:14.304611 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3d3bb35a-73c8-41c7-9a4c-33292a599224 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:14.412549 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:14.451203 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3bb35a-73c8-41c7-9a4c-33292a599224 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:14.466911 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c81bd-a852-4aa9-9ca1-12e9839d7037 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:14.468148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c81bd-a852-4aa9-9ca1-12e9839d7037 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:14.468633 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c81bd-a852-4aa9-9ca1-12e9839d7037 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:14.469203 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c81bd-a852-4aa9-9ca1-12e9839d7037 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:14.611993 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6c81bd-a852-4aa9-9ca1-12e9839d7037 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:14.615065 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6c81bd-a852-4aa9-9ca1-12e9839d7037 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:14.634981 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 917/1860] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:30:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:30:24.264559 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=62 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:24.353257 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a6e6d-d9f5-4c6c-af3a-bcd03c84e0fc None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:24.359198 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-637a6e6d-d9f5-4c6c-af3a-bcd03c84e0fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:24.501474 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a6e6d-d9f5-4c6c-af3a-bcd03c84e0fc None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:24.509181 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea999aef-82d0-464d-a880-8cc57cbdca72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:24.511356 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea999aef-82d0-464d-a880-8cc57cbdca72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:24.511356 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea999aef-82d0-464d-a880-8cc57cbdca72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:24.511826 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea999aef-82d0-464d-a880-8cc57cbdca72 admin admin] PATH_INFO: `/v3/users/9616e0d220134980aa955dd453af36a4` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:24.514111 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea999aef-82d0-464d-a880-8cc57cbdca72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9616e0d220134980aa955dd453af36a4)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:24.518497 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea999aef-82d0-464d-a880-8cc57cbdca72 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:24.626190 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=28 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:24.661162 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-ea999aef-82d0-464d-a880-8cc57cbdca72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9616e0d220134980aa955dd453af36a4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:24.687252 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-ea999aef-82d0-464d-a880-8cc57cbdca72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9616e0d220134980aa955dd453af36a4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:24.689421 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-ea999aef-82d0-464d-a880-8cc57cbdca72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9616e0d220134980aa955dd453af36a4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:24.691579 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-ea999aef-82d0-464d-a880-8cc57cbdca72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9616e0d220134980aa955dd453af36a4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:24.714027 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-ea999aef-82d0-464d-a880-8cc57cbdca72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9616e0d220134980aa955dd453af36a4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:24.750858 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 944/1861] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:30:24 2022] DELETE /identity/v3/users/9616e0d220134980aa955dd453af36a4 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:24.759788 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b3a24-6ee3-42f6-80c8-9d4cb94e7ac4 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:24.760634 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b3a24-6ee3-42f6-80c8-9d4cb94e7ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:24.761916 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b3a24-6ee3-42f6-80c8-9d4cb94e7ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:24.830763 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3e8b3a24-6ee3-42f6-80c8-9d4cb94e7ac4 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:24.861894 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3e8b3a24-6ee3-42f6-80c8-9d4cb94e7ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:25.154417 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 918/1862] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 397 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:25.177223 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea999aef-82d0-464d-a880-8cc57cbdca72 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:25.177223 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ea999aef-82d0-464d-a880-8cc57cbdca72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:25.296450 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea999aef-82d0-464d-a880-8cc57cbdca72 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:25.301876 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0abcb7-f6bf-42a3-a6ea-4bb0042b501f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:25.302627 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0abcb7-f6bf-42a3-a6ea-4bb0042b501f service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:25.303052 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0abcb7-f6bf-42a3-a6ea-4bb0042b501f service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:25.305698 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0abcb7-f6bf-42a3-a6ea-4bb0042b501f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:25.353313 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0abcb7-f6bf-42a3-a6ea-4bb0042b501f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:25.356853 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0abcb7-f6bf-42a3-a6ea-4bb0042b501f service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:25.377943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 945/1863] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:30:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:30:25.541834 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8b3a24-6ee3-42f6-80c8-9d4cb94e7ac4 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:25.544188 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3e8b3a24-6ee3-42f6-80c8-9d4cb94e7ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:25.560826 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8b3a24-6ee3-42f6-80c8-9d4cb94e7ac4 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:25.567894 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade446e-1876-45ba-a819-dc11ac385701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:25.569564 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade446e-1876-45ba-a819-dc11ac385701 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:25.569564 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade446e-1876-45ba-a819-dc11ac385701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:25.570121 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade446e-1876-45ba-a819-dc11ac385701 admin admin] PATH_INFO: `/v3/users/032838ae97784ca594e4f8ba8a97510f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:25.573247 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bade446e-1876-45ba-a819-dc11ac385701 admin admin] RBAC: Authorizing `identity:delete_user(user_id=032838ae97784ca594e4f8ba8a97510f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:25.574881 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bade446e-1876-45ba-a819-dc11ac385701 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:25.780772 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-bade446e-1876-45ba-a819-dc11ac385701 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '032838ae97784ca594e4f8ba8a97510f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:25.810490 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-bade446e-1876-45ba-a819-dc11ac385701 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '032838ae97784ca594e4f8ba8a97510f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:25.818082 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-bade446e-1876-45ba-a819-dc11ac385701 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '032838ae97784ca594e4f8ba8a97510f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:25.820107 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-bade446e-1876-45ba-a819-dc11ac385701 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '032838ae97784ca594e4f8ba8a97510f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:25.869322 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-bade446e-1876-45ba-a819-dc11ac385701 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '032838ae97784ca594e4f8ba8a97510f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:25.948692 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 919/1864] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:30:25 2022] DELETE /identity/v3/users/032838ae97784ca594e4f8ba8a97510f => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:25.954598 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022141f-f562-4e59-83ff-10c4c9b4ff78 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:25.955107 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022141f-f562-4e59-83ff-10c4c9b4ff78 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:25.955592 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022141f-f562-4e59-83ff-10c4c9b4ff78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:26.023439 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-2022141f-f562-4e59-83ff-10c4c9b4ff78 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:26.025619 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-2022141f-f562-4e59-83ff-10c4c9b4ff78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:26.070926 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:26.328722 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 946/1865] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:26.377288 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade446e-1876-45ba-a819-dc11ac385701 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:26.380187 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-bade446e-1876-45ba-a819-dc11ac385701 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:26.568996 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade446e-1876-45ba-a819-dc11ac385701 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:26.585977 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0c989a-5b37-455d-99cb-2e6bbb6d392a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:26.587339 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c989a-5b37-455d-99cb-2e6bbb6d392a service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:26.587855 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c989a-5b37-455d-99cb-2e6bbb6d392a service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:26.588346 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c989a-5b37-455d-99cb-2e6bbb6d392a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:26.610431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0c989a-5b37-455d-99cb-2e6bbb6d392a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:26.613364 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0c989a-5b37-455d-99cb-2e6bbb6d392a service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:26.642842 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 920/1866] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:30:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:30:26.780490 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2022141f-f562-4e59-83ff-10c4c9b4ff78 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:26.784892 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-2022141f-f562-4e59-83ff-10c4c9b4ff78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:26.814120 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2022141f-f562-4e59-83ff-10c4c9b4ff78 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:26.826154 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbf91b7-8ddd-42f6-a8b1-bca9a2cfb3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:26.828016 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf91b7-8ddd-42f6-a8b1-bca9a2cfb3dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:26.829840 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf91b7-8ddd-42f6-a8b1-bca9a2cfb3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:26.829840 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf91b7-8ddd-42f6-a8b1-bca9a2cfb3dc admin admin] PATH_INFO: `/v3/projects/a6ab8d0c96c44844a463d38d9b001032` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:26.832075 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbf91b7-8ddd-42f6-a8b1-bca9a2cfb3dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6ab8d0c96c44844a463d38d9b001032)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:26.836422 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbf91b7-8ddd-42f6-a8b1-bca9a2cfb3dc admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:27.011863 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8cbf91b7-8ddd-42f6-a8b1-bca9a2cfb3dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6ab8d0c96c44844a463d38d9b001032', 'request_id': 'req-8cbf91b7-8ddd-42f6-a8b1-bca9a2cfb3dc'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:27.046568 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8cbf91b7-8ddd-42f6-a8b1-bca9a2cfb3dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6ab8d0c96c44844a463d38d9b001032', 'request_id': 'req-8cbf91b7-8ddd-42f6-a8b1-bca9a2cfb3dc'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:27.081434 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8cbf91b7-8ddd-42f6-a8b1-bca9a2cfb3dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6ab8d0c96c44844a463d38d9b001032', 'request_id': 'req-8cbf91b7-8ddd-42f6-a8b1-bca9a2cfb3dc'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:27.114501 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8cbf91b7-8ddd-42f6-a8b1-bca9a2cfb3dc admin admin] The token cache is being invalidate because project a6ab8d0c96c44844a463d38d9b001032 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:30:27.114790 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8cbf91b7-8ddd-42f6-a8b1-bca9a2cfb3dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:27.119276 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8cbf91b7-8ddd-42f6-a8b1-bca9a2cfb3dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:27.126314 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 947/1867] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:30:26 2022] DELETE /identity/v3/projects/a6ab8d0c96c44844a463d38d9b001032 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:27.133075 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0c989a-5b37-455d-99cb-2e6bbb6d392a service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:27.165313 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0c989a-5b37-455d-99cb-2e6bbb6d392a service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:27.193351 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:27.313481 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3938e0-b13c-41b0-926c-b8d908eef901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:27.315700 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3938e0-b13c-41b0-926c-b8d908eef901 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:27.315893 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3938e0-b13c-41b0-926c-b8d908eef901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:27.316266 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3938e0-b13c-41b0-926c-b8d908eef901 admin admin] PATH_INFO: `/v3/users/66d9f6e69fa746489b5821b5a7a7eceb` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:27.318807 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3938e0-b13c-41b0-926c-b8d908eef901 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66d9f6e69fa746489b5821b5a7a7eceb)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:27.322901 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3938e0-b13c-41b0-926c-b8d908eef901 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:27.417850 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7c3938e0-b13c-41b0-926c-b8d908eef901 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66d9f6e69fa746489b5821b5a7a7eceb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:27.448279 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7c3938e0-b13c-41b0-926c-b8d908eef901 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66d9f6e69fa746489b5821b5a7a7eceb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:27.450490 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7c3938e0-b13c-41b0-926c-b8d908eef901 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66d9f6e69fa746489b5821b5a7a7eceb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:27.452250 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7c3938e0-b13c-41b0-926c-b8d908eef901 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66d9f6e69fa746489b5821b5a7a7eceb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:27.487153 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7c3938e0-b13c-41b0-926c-b8d908eef901 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66d9f6e69fa746489b5821b5a7a7eceb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:27.531791 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 921/1868] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:30:27 2022] DELETE /identity/v3/users/66d9f6e69fa746489b5821b5a7a7eceb => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:30:27.727196 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbf91b7-8ddd-42f6-a8b1-bca9a2cfb3dc admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:27.730928 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8cbf91b7-8ddd-42f6-a8b1-bca9a2cfb3dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:27.805905 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbf91b7-8ddd-42f6-a8b1-bca9a2cfb3dc admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:28.044672 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dfccb6-3fdf-4385-b631-8768b4c51536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:28.049410 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dfccb6-3fdf-4385-b631-8768b4c51536 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:28.049603 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dfccb6-3fdf-4385-b631-8768b4c51536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:28.050070 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dfccb6-3fdf-4385-b631-8768b4c51536 admin admin] PATH_INFO: `/v3/projects/42bcce2e8692420e97475ebbec0246ab` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:28.064748 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dfccb6-3fdf-4385-b631-8768b4c51536 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42bcce2e8692420e97475ebbec0246ab)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:28.077084 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dfccb6-3fdf-4385-b631-8768b4c51536 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:28.351881 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:28.358758 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-a8dfccb6-3fdf-4385-b631-8768b4c51536 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42bcce2e8692420e97475ebbec0246ab', 'request_id': 'req-a8dfccb6-3fdf-4385-b631-8768b4c51536'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:28.402165 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-a8dfccb6-3fdf-4385-b631-8768b4c51536 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42bcce2e8692420e97475ebbec0246ab', 'request_id': 'req-a8dfccb6-3fdf-4385-b631-8768b4c51536'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:28.426975 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-a8dfccb6-3fdf-4385-b631-8768b4c51536 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42bcce2e8692420e97475ebbec0246ab', 'request_id': 'req-a8dfccb6-3fdf-4385-b631-8768b4c51536'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:28.499411 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-a8dfccb6-3fdf-4385-b631-8768b4c51536 admin admin] The token cache is being invalidate because project 42bcce2e8692420e97475ebbec0246ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:30:28.500117 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-a8dfccb6-3fdf-4385-b631-8768b4c51536 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:28.501591 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-a8dfccb6-3fdf-4385-b631-8768b4c51536 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:28.519157 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 948/1869] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:30:27 2022] DELETE /identity/v3/projects/42bcce2e8692420e97475ebbec0246ab => generated 0 bytes in 792 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:28.526027 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3938e0-b13c-41b0-926c-b8d908eef901 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:28.530366 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7c3938e0-b13c-41b0-926c-b8d908eef901 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:28.582721 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3938e0-b13c-41b0-926c-b8d908eef901 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:28.675640 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dfccb6-3fdf-4385-b631-8768b4c51536 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:28.679557 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a8dfccb6-3fdf-4385-b631-8768b4c51536 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:28.750730 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f593e1a-cea2-4785-b5b1-117741dda509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:28.752752 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f593e1a-cea2-4785-b5b1-117741dda509 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:28.753385 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f593e1a-cea2-4785-b5b1-117741dda509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:28.753999 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f593e1a-cea2-4785-b5b1-117741dda509 admin admin] PATH_INFO: `/v3/users/bdfe4d53c10a4d57844e819a239b83c3` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:28.757847 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f593e1a-cea2-4785-b5b1-117741dda509 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdfe4d53c10a4d57844e819a239b83c3)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:28.762484 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f593e1a-cea2-4785-b5b1-117741dda509 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:28.912077 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dfccb6-3fdf-4385-b631-8768b4c51536 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:28.920297 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e132bb89-b358-4b87-ac95-02a0b4cdfbda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:28.922954 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e132bb89-b358-4b87-ac95-02a0b4cdfbda service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:28.923405 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:28.923975 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e132bb89-b358-4b87-ac95-02a0b4cdfbda service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:28.924680 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e132bb89-b358-4b87-ac95-02a0b4cdfbda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:28.929037 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e132bb89-b358-4b87-ac95-02a0b4cdfbda service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:28.959492 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8f593e1a-cea2-4785-b5b1-117741dda509 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdfe4d53c10a4d57844e819a239b83c3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:28.985972 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8f593e1a-cea2-4785-b5b1-117741dda509 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdfe4d53c10a4d57844e819a239b83c3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:28.988823 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8f593e1a-cea2-4785-b5b1-117741dda509 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdfe4d53c10a4d57844e819a239b83c3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:28.991871 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8f593e1a-cea2-4785-b5b1-117741dda509 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdfe4d53c10a4d57844e819a239b83c3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:29.019165 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8f593e1a-cea2-4785-b5b1-117741dda509 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdfe4d53c10a4d57844e819a239b83c3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:29.042696 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 922/1870] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:30:28 2022] DELETE /identity/v3/users/bdfe4d53c10a4d57844e819a239b83c3 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:29.043914 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f593e1a-cea2-4785-b5b1-117741dda509 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:29.046583 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8f593e1a-cea2-4785-b5b1-117741dda509 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:29.096901 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e132bb89-b358-4b87-ac95-02a0b4cdfbda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:29.100392 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e132bb89-b358-4b87-ac95-02a0b4cdfbda service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:29.125308 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 949/1871] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:30:28 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:30:29.235645 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f593e1a-cea2-4785-b5b1-117741dda509 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:29.248497 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639697d7-db16-4fef-b063-0e6310cc7c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:29.251145 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639697d7-db16-4fef-b063-0e6310cc7c80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:29.252220 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639697d7-db16-4fef-b063-0e6310cc7c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:29.253017 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639697d7-db16-4fef-b063-0e6310cc7c80 admin admin] PATH_INFO: `/v3/projects/4c877ece2ebe43138b7b83661eee5392` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:29.257226 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639697d7-db16-4fef-b063-0e6310cc7c80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c877ece2ebe43138b7b83661eee5392)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:29.261793 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639697d7-db16-4fef-b063-0e6310cc7c80 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:29.365898 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-639697d7-db16-4fef-b063-0e6310cc7c80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c877ece2ebe43138b7b83661eee5392', 'request_id': 'req-639697d7-db16-4fef-b063-0e6310cc7c80'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:29.382657 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-639697d7-db16-4fef-b063-0e6310cc7c80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c877ece2ebe43138b7b83661eee5392', 'request_id': 'req-639697d7-db16-4fef-b063-0e6310cc7c80'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:29.398335 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-639697d7-db16-4fef-b063-0e6310cc7c80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c877ece2ebe43138b7b83661eee5392', 'request_id': 'req-639697d7-db16-4fef-b063-0e6310cc7c80'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:29.419961 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-639697d7-db16-4fef-b063-0e6310cc7c80 admin admin] The token cache is being invalidate because project 4c877ece2ebe43138b7b83661eee5392 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:30:29.420462 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-639697d7-db16-4fef-b063-0e6310cc7c80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:29.422490 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-639697d7-db16-4fef-b063-0e6310cc7c80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:29.424894 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 923/1872] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:30:29 2022] DELETE /identity/v3/projects/4c877ece2ebe43138b7b83661eee5392 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:29.431622 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a737d6ad-4703-46fa-8bf0-25ca987c100f None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:29.432096 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a737d6ad-4703-46fa-8bf0-25ca987c100f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:29.432583 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a737d6ad-4703-46fa-8bf0-25ca987c100f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:29.502497 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-a737d6ad-4703-46fa-8bf0-25ca987c100f None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:29.505056 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a737d6ad-4703-46fa-8bf0-25ca987c100f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:29.712907 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639697d7-db16-4fef-b063-0e6310cc7c80 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:29.723414 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-639697d7-db16-4fef-b063-0e6310cc7c80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:29.757745 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 950/1873] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:30:29.764194 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7741c15-58b1-4724-9138-40cb74249b9a None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:29.764747 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7741c15-58b1-4724-9138-40cb74249b9a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:29.767380 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7741c15-58b1-4724-9138-40cb74249b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:29.813543 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639697d7-db16-4fef-b063-0e6310cc7c80 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:29.869612 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f49be1-0e8b-4da3-baa0-443c0e40fbc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:29.871547 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f49be1-0e8b-4da3-baa0-443c0e40fbc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:29.872070 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f49be1-0e8b-4da3-baa0-443c0e40fbc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:29.872573 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f49be1-0e8b-4da3-baa0-443c0e40fbc9 admin admin] PATH_INFO: `/v3/projects/67799fb01d1c464dbf5944da08298b4c` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:29.892143 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f49be1-0e8b-4da3-baa0-443c0e40fbc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67799fb01d1c464dbf5944da08298b4c)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:29.899017 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-d7741c15-58b1-4724-9138-40cb74249b9a None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:29.903732 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f49be1-0e8b-4da3-baa0-443c0e40fbc9 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:29.909387 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d7741c15-58b1-4724-9138-40cb74249b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:29.988367 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 951/1874] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:30:29.996097 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7741c15-58b1-4724-9138-40cb74249b9a None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:30.005366 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:30.020073 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d7f49be1-0e8b-4da3-baa0-443c0e40fbc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67799fb01d1c464dbf5944da08298b4c', 'request_id': 'req-d7f49be1-0e8b-4da3-baa0-443c0e40fbc9'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:30.037022 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d7f49be1-0e8b-4da3-baa0-443c0e40fbc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67799fb01d1c464dbf5944da08298b4c', 'request_id': 'req-d7f49be1-0e8b-4da3-baa0-443c0e40fbc9'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:30.044125 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7741c15-58b1-4724-9138-40cb74249b9a None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:30.059312 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d7f49be1-0e8b-4da3-baa0-443c0e40fbc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67799fb01d1c464dbf5944da08298b4c', 'request_id': 'req-d7f49be1-0e8b-4da3-baa0-443c0e40fbc9'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:30.096849 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:30.097342 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d7f49be1-0e8b-4da3-baa0-443c0e40fbc9 admin admin] The token cache is being invalidate because project 67799fb01d1c464dbf5944da08298b4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:30:30.099573 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d7f49be1-0e8b-4da3-baa0-443c0e40fbc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:30.100730 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d7f49be1-0e8b-4da3-baa0-443c0e40fbc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:30.105356 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 924/1875] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:30:29 2022] DELETE /identity/v3/projects/67799fb01d1c464dbf5944da08298b4c => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:30:30.117308 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1449af70-f8b1-4262-a2a8-2e6230d1e3b7 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:30.118080 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1449af70-f8b1-4262-a2a8-2e6230d1e3b7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:30.118684 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1449af70-f8b1-4262-a2a8-2e6230d1e3b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:30.223935 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-1449af70-f8b1-4262-a2a8-2e6230d1e3b7 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:30.225071 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1449af70-f8b1-4262-a2a8-2e6230d1e3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:30.376752 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db4c353-175c-4cc0-8877-a701be2522e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:30.377808 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4c353-175c-4cc0-8877-a701be2522e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:30.378425 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4c353-175c-4cc0-8877-a701be2522e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:30.378824 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4c353-175c-4cc0-8877-a701be2522e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:30.380171 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db4c353-175c-4cc0-8877-a701be2522e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:30:30.380596 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db4c353-175c-4cc0-8877-a701be2522e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:30.383468 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db4c353-175c-4cc0-8877-a701be2522e2 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:30.417701 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 952/1876] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:30:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 423 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:30:30.424424 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db4c353-175c-4cc0-8877-a701be2522e2 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:30.426155 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-6db4c353-175c-4cc0-8877-a701be2522e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:30.487817 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db4c353-175c-4cc0-8877-a701be2522e2 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:30.503612 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bbc1ff-6389-4ff3-bf7c-5ccac1ac3d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:30.504931 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bbc1ff-6389-4ff3-bf7c-5ccac1ac3d50 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:30.505373 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bbc1ff-6389-4ff3-bf7c-5ccac1ac3d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:30.505853 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bbc1ff-6389-4ff3-bf7c-5ccac1ac3d50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:30.506732 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bbc1ff-6389-4ff3-bf7c-5ccac1ac3d50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:30.509029 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bbc1ff-6389-4ff3-bf7c-5ccac1ac3d50 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:30.604625 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 953/1877] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:30:30 2022] POST /identity/v3/projects => generated 373 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:30:30.608887 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bbc1ff-6389-4ff3-bf7c-5ccac1ac3d50 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:30.617431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 925/1878] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 505 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:30.639847 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de1fa3-4f5d-4d1d-b367-b53b03cbea5f None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:30.643860 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de1fa3-4f5d-4d1d-b367-b53b03cbea5f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:30.643860 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de1fa3-4f5d-4d1d-b367-b53b03cbea5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:30.709802 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-08de1fa3-4f5d-4d1d-b367-b53b03cbea5f None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:30.711338 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-08de1fa3-4f5d-4d1d-b367-b53b03cbea5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:30.823939 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bbc1ff-6389-4ff3-bf7c-5ccac1ac3d50 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:30.841219 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da683ad-7e28-4d2e-8dc4-512abee2691d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:30.843154 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da683ad-7e28-4d2e-8dc4-512abee2691d admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:30.844011 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da683ad-7e28-4d2e-8dc4-512abee2691d admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:30.844492 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da683ad-7e28-4d2e-8dc4-512abee2691d admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:30.847111 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da683ad-7e28-4d2e-8dc4-512abee2691d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:30.853241 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da683ad-7e28-4d2e-8dc4-512abee2691d admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:30.886391 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 926/1879] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:30.894504 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08de1fa3-4f5d-4d1d-b367-b53b03cbea5f None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:30.907395 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08de1fa3-4f5d-4d1d-b367-b53b03cbea5f None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:30.912136 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 954/1880] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:30:30 2022] POST /identity/v3/users => generated 287 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:30:30.914582 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da683ad-7e28-4d2e-8dc4-512abee2691d admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:30.914901 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fee76f-cccd-443e-8a78-b69e1bc45178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:30.915566 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fee76f-cccd-443e-8a78-b69e1bc45178 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:30.915857 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fee76f-cccd-443e-8a78-b69e1bc45178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:30.916138 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fee76f-cccd-443e-8a78-b69e1bc45178 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:30.916924 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fee76f-cccd-443e-8a78-b69e1bc45178 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:30:30.917235 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fee76f-cccd-443e-8a78-b69e1bc45178 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:30.919502 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fee76f-cccd-443e-8a78-b69e1bc45178 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:30.945677 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da683ad-7e28-4d2e-8dc4-512abee2691d admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:30.950339 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 927/1881] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:30:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:30:30.952647 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fee76f-cccd-443e-8a78-b69e1bc45178 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:30.958855 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5ae8ac-a2e5-48fe-9019-6d965b36ca5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:30.959970 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5ae8ac-a2e5-48fe-9019-6d965b36ca5d admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:30.960468 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5ae8ac-a2e5-48fe-9019-6d965b36ca5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:30.960940 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5ae8ac-a2e5-48fe-9019-6d965b36ca5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:30.962649 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5ae8ac-a2e5-48fe-9019-6d965b36ca5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:30.969953 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5ae8ac-a2e5-48fe-9019-6d965b36ca5d admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:30.973118 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fee76f-cccd-443e-8a78-b69e1bc45178 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:30.984568 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8d1bb2-4365-44b0-9a64-1b8a8763afd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:30.984568 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d1bb2-4365-44b0-9a64-1b8a8763afd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:30.984568 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d1bb2-4365-44b0-9a64-1b8a8763afd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:30.985136 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d1bb2-4365-44b0-9a64-1b8a8763afd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:30.985769 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8d1bb2-4365-44b0-9a64-1b8a8763afd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:30.988182 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8d1bb2-4365-44b0-9a64-1b8a8763afd4 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:31.009048 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 955/1882] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:30 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:30:31.016063 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5ae8ac-a2e5-48fe-9019-6d965b36ca5d admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:31.043309 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5ae8ac-a2e5-48fe-9019-6d965b36ca5d admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:31.054799 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 928/1883] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:30:30 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:30:31.056234 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db61273d-c6e0-44e6-b141-eadb0882f455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:31.059535 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8d1bb2-4365-44b0-9a64-1b8a8763afd4 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:31.059724 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61273d-c6e0-44e6-b141-eadb0882f455 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:31.060118 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61273d-c6e0-44e6-b141-eadb0882f455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:31.060619 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61273d-c6e0-44e6-b141-eadb0882f455 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:31.066684 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db61273d-c6e0-44e6-b141-eadb0882f455 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:31.070446 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db61273d-c6e0-44e6-b141-eadb0882f455 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:31.110393 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 956/1884] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:31 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:30:31.112312 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db61273d-c6e0-44e6-b141-eadb0882f455 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:31.134816 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db61273d-c6e0-44e6-b141-eadb0882f455 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:31.157744 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:31.219802 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8d1bb2-4365-44b0-9a64-1b8a8763afd4 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:31.231931 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:31.235110 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0612d78-6d58-466d-9f04-54723daa4ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:31.237348 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0612d78-6d58-466d-9f04-54723daa4ce3 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:31.238343 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0612d78-6d58-466d-9f04-54723daa4ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:31.240227 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0612d78-6d58-466d-9f04-54723daa4ce3 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:31.242836 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0612d78-6d58-466d-9f04-54723daa4ce3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:31.247145 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9416a8-6e32-49fd-8673-5809cdc07c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:31.248148 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9416a8-6e32-49fd-8673-5809cdc07c2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:31.248625 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9416a8-6e32-49fd-8673-5809cdc07c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:31.249206 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9416a8-6e32-49fd-8673-5809cdc07c2c admin admin] PATH_INFO: `/v3/projects/681f57f09ffe48139ab8a13d82c75e7c/users/81f06c6a7cae43c5b947e6f2e616cd39/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:31.251340 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0612d78-6d58-466d-9f04-54723daa4ce3 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:31.282914 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9416a8-6e32-49fd-8673-5809cdc07c2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=681f57f09ffe48139ab8a13d82c75e7c, user_id=81f06c6a7cae43c5b947e6f2e616cd39, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:31.286958 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9416a8-6e32-49fd-8673-5809cdc07c2c admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:31.312723 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 957/1885] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:30:31 2022] PUT /identity/v3/projects/681f57f09ffe48139ab8a13d82c75e7c/users/81f06c6a7cae43c5b947e6f2e616cd39/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:31.321257 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 929/1886] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:30:31 2022] POST /identity/v3/users => generated 294 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:30:31.322791 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9416a8-6e32-49fd-8673-5809cdc07c2c admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:31.337273 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89003ae-2608-4165-87e0-0aa182c1e711 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:31.338313 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89003ae-2608-4165-87e0-0aa182c1e711 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:31.339673 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89003ae-2608-4165-87e0-0aa182c1e711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:31.407581 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-e89003ae-2608-4165-87e0-0aa182c1e711 None None] MFA Rules not processed for user `81f06c6a7cae43c5b947e6f2e616cd39`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:31.410424 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e89003ae-2608-4165-87e0-0aa182c1e711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:31.549362 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9416a8-6e32-49fd-8673-5809cdc07c2c admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:31.582433 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218fc51b-5c92-4133-ac8c-80dc9a7572dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:31.586036 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218fc51b-5c92-4133-ac8c-80dc9a7572dd admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:31.587041 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218fc51b-5c92-4133-ac8c-80dc9a7572dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:31.588156 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218fc51b-5c92-4133-ac8c-80dc9a7572dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:31.589949 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218fc51b-5c92-4133-ac8c-80dc9a7572dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:31.592730 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218fc51b-5c92-4133-ac8c-80dc9a7572dd admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:31.633954 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 958/1887] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:31 2022] GET /identity/v3/roles => generated 1420 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:30:31.638585 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218fc51b-5c92-4133-ac8c-80dc9a7572dd admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:31.639045 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 930/1888] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:31.664507 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89003ae-2608-4165-87e0-0aa182c1e711 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:31.666091 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e89003ae-2608-4165-87e0-0aa182c1e711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:31.678343 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218fc51b-5c92-4133-ac8c-80dc9a7572dd admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:31.693155 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f467a5c-1d1e-4b71-8be3-1de9cf6d3c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:31.693883 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f467a5c-1d1e-4b71-8be3-1de9cf6d3c57 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:31.694170 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f467a5c-1d1e-4b71-8be3-1de9cf6d3c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:31.694450 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f467a5c-1d1e-4b71-8be3-1de9cf6d3c57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:31.695207 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f467a5c-1d1e-4b71-8be3-1de9cf6d3c57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:31.701774 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f467a5c-1d1e-4b71-8be3-1de9cf6d3c57 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:31.729605 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 959/1889] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:31 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:30:31.741622 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f467a5c-1d1e-4b71-8be3-1de9cf6d3c57 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:31.759954 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f467a5c-1d1e-4b71-8be3-1de9cf6d3c57 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:31.778741 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaab89b-a8a5-45ac-8b4d-f79df05696c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:31.779583 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaab89b-a8a5-45ac-8b4d-f79df05696c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:31.779837 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaab89b-a8a5-45ac-8b4d-f79df05696c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:31.780199 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaab89b-a8a5-45ac-8b4d-f79df05696c2 admin admin] PATH_INFO: `/v3/projects/9fa84fa8f28e433f9844d18838009fae/users/742f16a660da42b1a3f6e8c834951dd3/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:31.812243 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89003ae-2608-4165-87e0-0aa182c1e711 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:31.819692 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaab89b-a8a5-45ac-8b4d-f79df05696c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fa84fa8f28e433f9844d18838009fae, user_id=742f16a660da42b1a3f6e8c834951dd3, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:31.840720 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaab89b-a8a5-45ac-8b4d-f79df05696c2 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:31.844652 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5779e904-6d01-4150-9936-322977b2c844 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:31.847144 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5779e904-6d01-4150-9936-322977b2c844 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:31.847537 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5779e904-6d01-4150-9936-322977b2c844 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:31.847864 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5779e904-6d01-4150-9936-322977b2c844 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:31.888264 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5779e904-6d01-4150-9936-322977b2c844 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:31.894889 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5779e904-6d01-4150-9936-322977b2c844 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:31.901326 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 960/1890] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:30:31 2022] PUT /identity/v3/projects/9fa84fa8f28e433f9844d18838009fae/users/742f16a660da42b1a3f6e8c834951dd3/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:31.917683 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 931/1891] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:30:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:30:31.930300 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02130282-d51c-44ba-afa8-a65b8f87f52d None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:31.931295 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02130282-d51c-44ba-afa8-a65b8f87f52d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:31.932068 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02130282-d51c-44ba-afa8-a65b8f87f52d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:32.055708 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-02130282-d51c-44ba-afa8-a65b8f87f52d None None] MFA Rules not processed for user `742f16a660da42b1a3f6e8c834951dd3`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:32.058100 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-02130282-d51c-44ba-afa8-a65b8f87f52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:32.283922 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 961/1892] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:32.290096 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5779e904-6d01-4150-9936-322977b2c844 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:32.334156 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5779e904-6d01-4150-9936-322977b2c844 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:32.552289 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:32.591865 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4f26f4-4079-48fb-9540-f77b6c797a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:32.595501 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f26f4-4079-48fb-9540-f77b6c797a04 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:32.596209 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f26f4-4079-48fb-9540-f77b6c797a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:32.596759 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f26f4-4079-48fb-9540-f77b6c797a04 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:32.599657 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4f26f4-4079-48fb-9540-f77b6c797a04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:32.604968 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4f26f4-4079-48fb-9540-f77b6c797a04 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:32.756126 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02130282-d51c-44ba-afa8-a65b8f87f52d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:32.758288 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 932/1893] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:30:32 2022] POST /identity/v3/projects => generated 387 bytes in 467 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:30:32.807611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02130282-d51c-44ba-afa8-a65b8f87f52d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:32.871561 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:32.969305 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4de701-191c-46c0-bc0c-b9af72acfb2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:32.969862 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4de701-191c-46c0-bc0c-b9af72acfb2c admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:32.969997 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4de701-191c-46c0-bc0c-b9af72acfb2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:32.970366 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4de701-191c-46c0-bc0c-b9af72acfb2c admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:32.971214 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4de701-191c-46c0-bc0c-b9af72acfb2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:32.973313 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4de701-191c-46c0-bc0c-b9af72acfb2c admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:33.043389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 962/1894] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:30:32 2022] POST /identity/v3/users => generated 300 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:30:33.048768 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4f26f4-4079-48fb-9540-f77b6c797a04 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:33.067550 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4f26f4-4079-48fb-9540-f77b6c797a04 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:33.085674 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18f253-22f9-4999-83fe-feaf04e09495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:33.088550 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18f253-22f9-4999-83fe-feaf04e09495 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:33.091518 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18f253-22f9-4999-83fe-feaf04e09495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:33.093942 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18f253-22f9-4999-83fe-feaf04e09495 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:33.095400 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d18f253-22f9-4999-83fe-feaf04e09495 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:33.104150 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d18f253-22f9-4999-83fe-feaf04e09495 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:33.132744 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 933/1895] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:33 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:30:33.144391 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4de701-191c-46c0-bc0c-b9af72acfb2c admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:33.180433 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4de701-191c-46c0-bc0c-b9af72acfb2c admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:33.203585 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df97fd34-a767-470e-8ca9-ef451510272b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:33.210832 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df97fd34-a767-470e-8ca9-ef451510272b admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:33.213043 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df97fd34-a767-470e-8ca9-ef451510272b admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:33.214675 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df97fd34-a767-470e-8ca9-ef451510272b admin admin] PATH_INFO: `/v3/projects/25f278d635b14c2b85956ae6042f9017/users/acf7b5fa2da340469f0c6e5deb579228/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:33.284966 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df97fd34-a767-470e-8ca9-ef451510272b admin admin] RBAC: Authorizing `identity:create_grant(project_id=25f278d635b14c2b85956ae6042f9017, user_id=acf7b5fa2da340469f0c6e5deb579228, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:33.291805 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df97fd34-a767-470e-8ca9-ef451510272b admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:33.327815 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 963/1896] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:30:33 2022] PUT /identity/v3/projects/25f278d635b14c2b85956ae6042f9017/users/acf7b5fa2da340469f0c6e5deb579228/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:33.334846 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18f253-22f9-4999-83fe-feaf04e09495 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:33.355127 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18f253-22f9-4999-83fe-feaf04e09495 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:33.508009 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee382938-81f6-4746-a877-bf4465c8e0b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:33.508730 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee382938-81f6-4746-a877-bf4465c8e0b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:33.509454 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee382938-81f6-4746-a877-bf4465c8e0b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:33.509865 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee382938-81f6-4746-a877-bf4465c8e0b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:33.511147 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee382938-81f6-4746-a877-bf4465c8e0b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:33.512983 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee382938-81f6-4746-a877-bf4465c8e0b8 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:33.534353 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 934/1897] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:33 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:30:33.545997 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df97fd34-a767-470e-8ca9-ef451510272b admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:33.575830 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df97fd34-a767-470e-8ca9-ef451510272b admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:33.586700 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c67d2b-deb9-48ad-8450-16fe35b2e1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:33.588551 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c67d2b-deb9-48ad-8450-16fe35b2e1bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:33.589101 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c67d2b-deb9-48ad-8450-16fe35b2e1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:33.589695 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c67d2b-deb9-48ad-8450-16fe35b2e1bf admin admin] PATH_INFO: `/v3/projects/25f278d635b14c2b85956ae6042f9017/users/acf7b5fa2da340469f0c6e5deb579228/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:33.594345 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c67d2b-deb9-48ad-8450-16fe35b2e1bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=25f278d635b14c2b85956ae6042f9017, user_id=acf7b5fa2da340469f0c6e5deb579228, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:33.600308 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c67d2b-deb9-48ad-8450-16fe35b2e1bf admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:33.615713 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 964/1898] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:30:33 2022] PUT /identity/v3/projects/25f278d635b14c2b85956ae6042f9017/users/acf7b5fa2da340469f0c6e5deb579228/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:33.635163 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843dfdc5-b0d4-4f4f-b857-d95c69cae877 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:33.635458 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843dfdc5-b0d4-4f4f-b857-d95c69cae877 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:33.635746 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843dfdc5-b0d4-4f4f-b857-d95c69cae877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:33.692331 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-843dfdc5-b0d4-4f4f-b857-d95c69cae877 None None] MFA Rules not processed for user `acf7b5fa2da340469f0c6e5deb579228`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:33.694557 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-843dfdc5-b0d4-4f4f-b857-d95c69cae877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:33.831735 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 935/1899] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:33 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:33.862597 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c67d2b-deb9-48ad-8450-16fe35b2e1bf admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:33.865009 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-15c67d2b-deb9-48ad-8450-16fe35b2e1bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:34.019512 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c67d2b-deb9-48ad-8450-16fe35b2e1bf admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:34.038353 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27933523-4948-4510-be06-bd197c8d2009 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:34.039052 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27933523-4948-4510-be06-bd197c8d2009 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:34.039527 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27933523-4948-4510-be06-bd197c8d2009 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:34.040022 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27933523-4948-4510-be06-bd197c8d2009 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:34.066139 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27933523-4948-4510-be06-bd197c8d2009 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:34.069918 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27933523-4948-4510-be06-bd197c8d2009 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:34.106736 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 965/1900] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:30:33 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:30:34.196967 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843dfdc5-b0d4-4f4f-b857-d95c69cae877 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:34.473997 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:34.543248 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843dfdc5-b0d4-4f4f-b857-d95c69cae877 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:34.562065 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17189567-a568-413a-9c55-0f44cabac3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:34.563688 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17189567-a568-413a-9c55-0f44cabac3d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:34.571642 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17189567-a568-413a-9c55-0f44cabac3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:34.572233 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17189567-a568-413a-9c55-0f44cabac3d0 admin admin] PATH_INFO: `/v3/users/742f16a660da42b1a3f6e8c834951dd3` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:34.578482 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17189567-a568-413a-9c55-0f44cabac3d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=742f16a660da42b1a3f6e8c834951dd3)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:34.582372 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17189567-a568-413a-9c55-0f44cabac3d0 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:34.775015 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-17189567-a568-413a-9c55-0f44cabac3d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '742f16a660da42b1a3f6e8c834951dd3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:34.792014 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-17189567-a568-413a-9c55-0f44cabac3d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '742f16a660da42b1a3f6e8c834951dd3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:34.793267 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-17189567-a568-413a-9c55-0f44cabac3d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '742f16a660da42b1a3f6e8c834951dd3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:34.805216 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-17189567-a568-413a-9c55-0f44cabac3d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '742f16a660da42b1a3f6e8c834951dd3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:34.868439 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-17189567-a568-413a-9c55-0f44cabac3d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '742f16a660da42b1a3f6e8c834951dd3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:34.912532 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 936/1901] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:30:34 2022] DELETE /identity/v3/users/742f16a660da42b1a3f6e8c834951dd3 => generated 0 bytes in 712 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:34.926456 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd1306c-bed6-4604-bbcf-d1e85bd7a34b None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:34.929122 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd1306c-bed6-4604-bbcf-d1e85bd7a34b None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:34.929343 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd1306c-bed6-4604-bbcf-d1e85bd7a34b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:35.001325 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-fcd1306c-bed6-4604-bbcf-d1e85bd7a34b None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:35.005227 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-fcd1306c-bed6-4604-bbcf-d1e85bd7a34b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:35.179496 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 966/1902] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:35.198282 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17189567-a568-413a-9c55-0f44cabac3d0 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:35.202072 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-17189567-a568-413a-9c55-0f44cabac3d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:35.377610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17189567-a568-413a-9c55-0f44cabac3d0 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:35.384917 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9db54d9-24ab-451e-b379-7eb877901e56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:35.386271 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db54d9-24ab-451e-b379-7eb877901e56 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:35.386786 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db54d9-24ab-451e-b379-7eb877901e56 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:35.387897 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db54d9-24ab-451e-b379-7eb877901e56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:35.411529 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9db54d9-24ab-451e-b379-7eb877901e56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:35.415558 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9db54d9-24ab-451e-b379-7eb877901e56 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:35.462115 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 937/1903] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:30:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:30:36.023864 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd1306c-bed6-4604-bbcf-d1e85bd7a34b None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:36.026035 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-fcd1306c-bed6-4604-bbcf-d1e85bd7a34b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:36.068041 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd1306c-bed6-4604-bbcf-d1e85bd7a34b None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:36.088105 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a9dbb1-17de-4814-bd00-6eb8f93c8caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:36.099663 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a9dbb1-17de-4814-bd00-6eb8f93c8caf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:36.101365 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a9dbb1-17de-4814-bd00-6eb8f93c8caf admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:36.101365 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a9dbb1-17de-4814-bd00-6eb8f93c8caf admin admin] PATH_INFO: `/v3/projects/9fa84fa8f28e433f9844d18838009fae` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:36.107606 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a9dbb1-17de-4814-bd00-6eb8f93c8caf admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fa84fa8f28e433f9844d18838009fae)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:36.118487 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a9dbb1-17de-4814-bd00-6eb8f93c8caf admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:36.164235 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:36.254678 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d2a9dbb1-17de-4814-bd00-6eb8f93c8caf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fa84fa8f28e433f9844d18838009fae', 'request_id': 'req-d2a9dbb1-17de-4814-bd00-6eb8f93c8caf'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:36.268022 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d2a9dbb1-17de-4814-bd00-6eb8f93c8caf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fa84fa8f28e433f9844d18838009fae', 'request_id': 'req-d2a9dbb1-17de-4814-bd00-6eb8f93c8caf'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:36.286547 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d2a9dbb1-17de-4814-bd00-6eb8f93c8caf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fa84fa8f28e433f9844d18838009fae', 'request_id': 'req-d2a9dbb1-17de-4814-bd00-6eb8f93c8caf'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:36.312580 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d2a9dbb1-17de-4814-bd00-6eb8f93c8caf admin admin] The token cache is being invalidate because project 9fa84fa8f28e433f9844d18838009fae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:30:36.313100 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d2a9dbb1-17de-4814-bd00-6eb8f93c8caf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:36.315051 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d2a9dbb1-17de-4814-bd00-6eb8f93c8caf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:36.318099 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 967/1904] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:30:36 2022] DELETE /identity/v3/projects/9fa84fa8f28e433f9844d18838009fae => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:30:36.324368 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9db54d9-24ab-451e-b379-7eb877901e56 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:36.349488 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9db54d9-24ab-451e-b379-7eb877901e56 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:36.491435 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a908a-107b-4ce6-a0dd-ef7b5775dcc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:36.493299 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a908a-107b-4ce6-a0dd-ef7b5775dcc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:36.493803 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a908a-107b-4ce6-a0dd-ef7b5775dcc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:36.494665 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a908a-107b-4ce6-a0dd-ef7b5775dcc3 admin admin] PATH_INFO: `/v3/users/acf7b5fa2da340469f0c6e5deb579228` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:36.498077 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7a908a-107b-4ce6-a0dd-ef7b5775dcc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acf7b5fa2da340469f0c6e5deb579228)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:36.502366 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7a908a-107b-4ce6-a0dd-ef7b5775dcc3 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:36.548158 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:36.627794 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0c7a908a-107b-4ce6-a0dd-ef7b5775dcc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acf7b5fa2da340469f0c6e5deb579228'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:36.648141 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0c7a908a-107b-4ce6-a0dd-ef7b5775dcc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acf7b5fa2da340469f0c6e5deb579228'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:36.649587 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0c7a908a-107b-4ce6-a0dd-ef7b5775dcc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acf7b5fa2da340469f0c6e5deb579228'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:36.651431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0c7a908a-107b-4ce6-a0dd-ef7b5775dcc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acf7b5fa2da340469f0c6e5deb579228'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:36.677787 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0c7a908a-107b-4ce6-a0dd-ef7b5775dcc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acf7b5fa2da340469f0c6e5deb579228'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:36.706160 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 938/1905] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:30:36 2022] DELETE /identity/v3/users/acf7b5fa2da340469f0c6e5deb579228 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:30:37.212745 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a9dbb1-17de-4814-bd00-6eb8f93c8caf admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:37.220688 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d2a9dbb1-17de-4814-bd00-6eb8f93c8caf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:37.258491 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a9dbb1-17de-4814-bd00-6eb8f93c8caf admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:37.371968 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc1f71d-0165-4edf-a57e-39fc2bc27a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:37.373437 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc1f71d-0165-4edf-a57e-39fc2bc27a64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:37.374196 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc1f71d-0165-4edf-a57e-39fc2bc27a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:37.374640 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc1f71d-0165-4edf-a57e-39fc2bc27a64 admin admin] PATH_INFO: `/v3/projects/25f278d635b14c2b85956ae6042f9017` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:37.376963 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc1f71d-0165-4edf-a57e-39fc2bc27a64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25f278d635b14c2b85956ae6042f9017)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:37.379716 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc1f71d-0165-4edf-a57e-39fc2bc27a64 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:37.471426 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0dc1f71d-0165-4edf-a57e-39fc2bc27a64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25f278d635b14c2b85956ae6042f9017', 'request_id': 'req-0dc1f71d-0165-4edf-a57e-39fc2bc27a64'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:37.483318 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0dc1f71d-0165-4edf-a57e-39fc2bc27a64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25f278d635b14c2b85956ae6042f9017', 'request_id': 'req-0dc1f71d-0165-4edf-a57e-39fc2bc27a64'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:37.497335 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0dc1f71d-0165-4edf-a57e-39fc2bc27a64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25f278d635b14c2b85956ae6042f9017', 'request_id': 'req-0dc1f71d-0165-4edf-a57e-39fc2bc27a64'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:37.522795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0dc1f71d-0165-4edf-a57e-39fc2bc27a64 admin admin] The token cache is being invalidate because project 25f278d635b14c2b85956ae6042f9017 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:30:37.523200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0dc1f71d-0165-4edf-a57e-39fc2bc27a64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:37.524306 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0dc1f71d-0165-4edf-a57e-39fc2bc27a64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:37.527796 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 968/1906] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:30:37 2022] DELETE /identity/v3/projects/25f278d635b14c2b85956ae6042f9017 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:37.533282 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26c1e5-8b4f-448e-b54e-b76484f0f120 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:37.534109 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26c1e5-8b4f-448e-b54e-b76484f0f120 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:37.534109 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26c1e5-8b4f-448e-b54e-b76484f0f120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:37.584582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-9d26c1e5-8b4f-448e-b54e-b76484f0f120 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:37.586782 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9d26c1e5-8b4f-448e-b54e-b76484f0f120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:37.738118 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 939/1907] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:30:37.750778 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac0a0b-719a-4918-966c-3c9df66833f2 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:37.751260 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac0a0b-719a-4918-966c-3c9df66833f2 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:37.751736 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac0a0b-719a-4918-966c-3c9df66833f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:37.791091 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-faac0a0b-719a-4918-966c-3c9df66833f2 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:37.793316 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-faac0a0b-719a-4918-966c-3c9df66833f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:37.813617 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 969/1908] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:37.817238 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d26c1e5-8b4f-448e-b54e-b76484f0f120 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:37.846613 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d26c1e5-8b4f-448e-b54e-b76484f0f120 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:37.878702 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586f99a5-c412-4644-93ee-f2531ad47b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:37.880274 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f99a5-c412-4644-93ee-f2531ad47b8b admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:37.880894 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f99a5-c412-4644-93ee-f2531ad47b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:37.881289 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f99a5-c412-4644-93ee-f2531ad47b8b admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:37.884133 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586f99a5-c412-4644-93ee-f2531ad47b8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:30:37.884797 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586f99a5-c412-4644-93ee-f2531ad47b8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:37.888397 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586f99a5-c412-4644-93ee-f2531ad47b8b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:37.921998 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac0a0b-719a-4918-966c-3c9df66833f2 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:37.932134 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 940/1909] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:30:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:30:37.940816 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac0a0b-719a-4918-966c-3c9df66833f2 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:37.953767 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b8744e-ef04-4292-af7b-c2e1b1882a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:37.955058 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8744e-ef04-4292-af7b-c2e1b1882a61 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:37.955849 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8744e-ef04-4292-af7b-c2e1b1882a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:37.956641 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8744e-ef04-4292-af7b-c2e1b1882a61 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:37.958121 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b8744e-ef04-4292-af7b-c2e1b1882a61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:37.962330 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b8744e-ef04-4292-af7b-c2e1b1882a61 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:38.054926 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 970/1910] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:30:37 2022] POST /identity/v3/projects => generated 385 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:30:38.064285 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586f99a5-c412-4644-93ee-f2531ad47b8b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:38.106611 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586f99a5-c412-4644-93ee-f2531ad47b8b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:38.365977 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921e1538-2db9-4f58-ad47-4e049964eb26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:38.366731 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e1538-2db9-4f58-ad47-4e049964eb26 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:38.367182 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e1538-2db9-4f58-ad47-4e049964eb26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:38.367536 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e1538-2db9-4f58-ad47-4e049964eb26 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:38.368276 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921e1538-2db9-4f58-ad47-4e049964eb26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:38.372153 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921e1538-2db9-4f58-ad47-4e049964eb26 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:38.411746 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 941/1911] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:30:38 2022] POST /identity/v3/users => generated 293 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:30:38.424915 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b8744e-ef04-4292-af7b-c2e1b1882a61 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:38.436817 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:38.447247 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b8744e-ef04-4292-af7b-c2e1b1882a61 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:38.463591 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e48eafb-2516-4a35-84f1-51a108b58895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:38.466125 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48eafb-2516-4a35-84f1-51a108b58895 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:38.466626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48eafb-2516-4a35-84f1-51a108b58895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:38.469717 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48eafb-2516-4a35-84f1-51a108b58895 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:38.472383 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e48eafb-2516-4a35-84f1-51a108b58895 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:38.476024 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e48eafb-2516-4a35-84f1-51a108b58895 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:38.517722 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 971/1912] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:38 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:30:38.521966 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921e1538-2db9-4f58-ad47-4e049964eb26 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:38.558221 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921e1538-2db9-4f58-ad47-4e049964eb26 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:38.570663 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af4826-336f-4e12-a00d-425b9c721b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:38.571917 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af4826-336f-4e12-a00d-425b9c721b09 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:38.573098 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af4826-336f-4e12-a00d-425b9c721b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:38.573098 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af4826-336f-4e12-a00d-425b9c721b09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:38.574783 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af4826-336f-4e12-a00d-425b9c721b09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:38.579467 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af4826-336f-4e12-a00d-425b9c721b09 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:38.584139 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:38.611207 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 942/1913] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:38 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:30:38.620057 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e48eafb-2516-4a35-84f1-51a108b58895 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:38.643621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e48eafb-2516-4a35-84f1-51a108b58895 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:38.651240 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a3021-d081-4c62-9482-dda45104489a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:38.652253 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a3021-d081-4c62-9482-dda45104489a admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:38.652499 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a3021-d081-4c62-9482-dda45104489a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:38.652764 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a3021-d081-4c62-9482-dda45104489a admin admin] PATH_INFO: `/v3/projects/fc7c501a92cb43a18fea59e18066d83a/users/67f34090b91d4b0cb5f7bd6f8db47de2/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:38.698062 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0a3021-d081-4c62-9482-dda45104489a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc7c501a92cb43a18fea59e18066d83a, user_id=67f34090b91d4b0cb5f7bd6f8db47de2, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:38.702018 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0a3021-d081-4c62-9482-dda45104489a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:38.737696 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 972/1914] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:30:38 2022] PUT /identity/v3/projects/fc7c501a92cb43a18fea59e18066d83a/users/67f34090b91d4b0cb5f7bd6f8db47de2/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:38.748220 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36991a-416c-4597-b2e6-b74859e91326 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:38.748916 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36991a-416c-4597-b2e6-b74859e91326 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:38.749460 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36991a-416c-4597-b2e6-b74859e91326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:38.934109 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-de36991a-416c-4597-b2e6-b74859e91326 None None] MFA Rules not processed for user `67f34090b91d4b0cb5f7bd6f8db47de2`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:38.949340 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-de36991a-416c-4597-b2e6-b74859e91326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:39.238358 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 943/1915] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 493 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:39.243150 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a3021-d081-4c62-9482-dda45104489a admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:39.597723 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a3021-d081-4c62-9482-dda45104489a admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:39.622015 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da678de-e9e5-4c2f-8d14-becc6a8d87b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:39.626753 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da678de-e9e5-4c2f-8d14-becc6a8d87b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:39.627811 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da678de-e9e5-4c2f-8d14-becc6a8d87b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:39.629486 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da678de-e9e5-4c2f-8d14-becc6a8d87b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:39.631654 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da678de-e9e5-4c2f-8d14-becc6a8d87b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:39.644049 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da678de-e9e5-4c2f-8d14-becc6a8d87b6 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:39.706719 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de36991a-416c-4597-b2e6-b74859e91326 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:39.714463 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-de36991a-416c-4597-b2e6-b74859e91326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:39.774525 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 973/1916] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:30:39 2022] POST /identity/v3/projects => generated 385 bytes in 535 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:30:39.781498 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da678de-e9e5-4c2f-8d14-becc6a8d87b6 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:39.816900 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de36991a-416c-4597-b2e6-b74859e91326 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:40.127375 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da678de-e9e5-4c2f-8d14-becc6a8d87b6 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:40.155430 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61866259-eed3-4a07-8bc2-2e9955882f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:40.156337 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08264290-624a-4cff-a488-8a1d9b8c8489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:40.156535 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61866259-eed3-4a07-8bc2-2e9955882f5a admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:40.156940 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61866259-eed3-4a07-8bc2-2e9955882f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:40.157341 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08264290-624a-4cff-a488-8a1d9b8c8489 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:40.157496 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61866259-eed3-4a07-8bc2-2e9955882f5a admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:40.158701 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61866259-eed3-4a07-8bc2-2e9955882f5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:40.159514 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08264290-624a-4cff-a488-8a1d9b8c8489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:40.160004 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08264290-624a-4cff-a488-8a1d9b8c8489 admin admin] PATH_INFO: `/v3/users/81f06c6a7cae43c5b947e6f2e616cd39` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:40.162396 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08264290-624a-4cff-a488-8a1d9b8c8489 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81f06c6a7cae43c5b947e6f2e616cd39)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:40.166754 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08264290-624a-4cff-a488-8a1d9b8c8489 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:40.171246 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61866259-eed3-4a07-8bc2-2e9955882f5a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:40.241297 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 974/1917] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:30:39 2022] POST /identity/v3/users => generated 293 bytes in 461 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:30:40.252205 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61866259-eed3-4a07-8bc2-2e9955882f5a admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:40.307581 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61866259-eed3-4a07-8bc2-2e9955882f5a admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:40.345268 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8b0bc6-bbf8-41b7-bc8e-41d6dec21af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:40.347024 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b0bc6-bbf8-41b7-bc8e-41d6dec21af7 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:40.347754 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b0bc6-bbf8-41b7-bc8e-41d6dec21af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:40.348498 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b0bc6-bbf8-41b7-bc8e-41d6dec21af7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:40.350215 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8b0bc6-bbf8-41b7-bc8e-41d6dec21af7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:40.351504 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-08264290-624a-4cff-a488-8a1d9b8c8489 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81f06c6a7cae43c5b947e6f2e616cd39'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:40.353874 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8b0bc6-bbf8-41b7-bc8e-41d6dec21af7 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:40.371248 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-08264290-624a-4cff-a488-8a1d9b8c8489 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81f06c6a7cae43c5b947e6f2e616cd39'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:40.395599 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-08264290-624a-4cff-a488-8a1d9b8c8489 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81f06c6a7cae43c5b947e6f2e616cd39'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:40.397191 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-08264290-624a-4cff-a488-8a1d9b8c8489 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81f06c6a7cae43c5b947e6f2e616cd39'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:40.427278 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 975/1918] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:30:40.428894 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8b0bc6-bbf8-41b7-bc8e-41d6dec21af7 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:40.432379 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-2b8b0bc6-bbf8-41b7-bc8e-41d6dec21af7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:40.465363 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8b0bc6-bbf8-41b7-bc8e-41d6dec21af7 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:40.468271 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-08264290-624a-4cff-a488-8a1d9b8c8489 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81f06c6a7cae43c5b947e6f2e616cd39'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:40.482004 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7140b4e4-d131-4bf0-84a1-f741324c1197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:40.485413 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140b4e4-d131-4bf0-84a1-f741324c1197 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:40.485413 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140b4e4-d131-4bf0-84a1-f741324c1197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:40.485413 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140b4e4-d131-4bf0-84a1-f741324c1197 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:40.485413 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7140b4e4-d131-4bf0-84a1-f741324c1197 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:40.488960 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7140b4e4-d131-4bf0-84a1-f741324c1197 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:40.529576 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 944/1919] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:30:39 2022] DELETE /identity/v3/users/81f06c6a7cae43c5b947e6f2e616cd39 => generated 0 bytes in 823 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:40.534981 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41bdf3-8b97-462d-ac16-26b12bdcdb92 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:40.535428 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41bdf3-8b97-462d-ac16-26b12bdcdb92 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:40.535859 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41bdf3-8b97-462d-ac16-26b12bdcdb92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:40.560541 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 976/1920] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:30:40.572592 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7140b4e4-d131-4bf0-84a1-f741324c1197 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:40.595772 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:40.616716 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7140b4e4-d131-4bf0-84a1-f741324c1197 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:40.622398 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-af41bdf3-8b97-462d-ac16-26b12bdcdb92 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:40.624447 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-af41bdf3-8b97-462d-ac16-26b12bdcdb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:40.694305 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:40.833705 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86186ba-d162-496d-84f7-789b8b4902ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:40.839330 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86186ba-d162-496d-84f7-789b8b4902ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:40.840309 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86186ba-d162-496d-84f7-789b8b4902ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:40.841052 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86186ba-d162-496d-84f7-789b8b4902ea admin admin] PATH_INFO: `/v3/projects/702d9d1b50ab480e9604807eebed8182/users/72c2735c521e452aba5189487d1ca3cc/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:40.888068 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86186ba-d162-496d-84f7-789b8b4902ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=702d9d1b50ab480e9604807eebed8182, user_id=72c2735c521e452aba5189487d1ca3cc, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:40.903276 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86186ba-d162-496d-84f7-789b8b4902ea admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:40.926240 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 945/1921] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 396 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:40.929287 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af41bdf3-8b97-462d-ac16-26b12bdcdb92 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:40.932268 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-af41bdf3-8b97-462d-ac16-26b12bdcdb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:40.985821 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af41bdf3-8b97-462d-ac16-26b12bdcdb92 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:40.986524 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 977/1922] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:30:40 2022] PUT /identity/v3/projects/702d9d1b50ab480e9604807eebed8182/users/72c2735c521e452aba5189487d1ca3cc/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:30:40.992926 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86186ba-d162-496d-84f7-789b8b4902ea admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:41.013754 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a86186ba-d162-496d-84f7-789b8b4902ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:41.205269 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e51118-e24e-40ac-ba24-1b313dbd71b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:41.207960 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e51118-e24e-40ac-ba24-1b313dbd71b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:41.209615 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e51118-e24e-40ac-ba24-1b313dbd71b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:41.210701 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e51118-e24e-40ac-ba24-1b313dbd71b3 admin admin] PATH_INFO: `/v3/users/cc4aca40ca4e4d48be6830e5ea067246` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:41.213888 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e51118-e24e-40ac-ba24-1b313dbd71b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc4aca40ca4e4d48be6830e5ea067246)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:41.219131 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e51118-e24e-40ac-ba24-1b313dbd71b3 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:41.222665 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86186ba-d162-496d-84f7-789b8b4902ea admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:41.227272 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0822e05e-b073-4247-8224-6a418c25a0b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:41.228143 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822e05e-b073-4247-8224-6a418c25a0b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:41.228323 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822e05e-b073-4247-8224-6a418c25a0b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:41.228745 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822e05e-b073-4247-8224-6a418c25a0b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:41.255933 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0822e05e-b073-4247-8224-6a418c25a0b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:41.258911 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0822e05e-b073-4247-8224-6a418c25a0b0 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:41.291236 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 978/1923] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:30:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:30:41.293830 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8572344e-b1db-4979-9f8d-fc178c148a93 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:41.294312 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8572344e-b1db-4979-9f8d-fc178c148a93 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:41.294914 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8572344e-b1db-4979-9f8d-fc178c148a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:41.362369 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-8572344e-b1db-4979-9f8d-fc178c148a93 None None] MFA Rules not processed for user `72c2735c521e452aba5189487d1ca3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:41.364825 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8572344e-b1db-4979-9f8d-fc178c148a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:41.378999 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-32e51118-e24e-40ac-ba24-1b313dbd71b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc4aca40ca4e4d48be6830e5ea067246'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:41.412857 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-32e51118-e24e-40ac-ba24-1b313dbd71b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc4aca40ca4e4d48be6830e5ea067246'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:41.415091 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-32e51118-e24e-40ac-ba24-1b313dbd71b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc4aca40ca4e4d48be6830e5ea067246'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:41.418540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-32e51118-e24e-40ac-ba24-1b313dbd71b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc4aca40ca4e4d48be6830e5ea067246'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:41.514186 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-32e51118-e24e-40ac-ba24-1b313dbd71b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc4aca40ca4e4d48be6830e5ea067246'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:41.564376 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 946/1924] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:30:40 2022] DELETE /identity/v3/users/cc4aca40ca4e4d48be6830e5ea067246 => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:41.572017 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23cd0cc-d79b-43a7-a9a1-2de741c83d18 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:41.572602 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23cd0cc-d79b-43a7-a9a1-2de741c83d18 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:41.573138 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23cd0cc-d79b-43a7-a9a1-2de741c83d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:41.632626 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b23cd0cc-d79b-43a7-a9a1-2de741c83d18 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:41.637266 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b23cd0cc-d79b-43a7-a9a1-2de741c83d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:41.718081 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 979/1925] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 426 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:41.722685 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:41.726273 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8572344e-b1db-4979-9f8d-fc178c148a93 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:41.733502 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8572344e-b1db-4979-9f8d-fc178c148a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:41.784234 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8572344e-b1db-4979-9f8d-fc178c148a93 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:41.878475 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 947/1926] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:41.894260 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:41.908012 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23cd0cc-d79b-43a7-a9a1-2de741c83d18 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:41.911495 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b23cd0cc-d79b-43a7-a9a1-2de741c83d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:41.980075 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64abfb2c-f8e1-427d-9440-afdff59bf728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:41.981488 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64abfb2c-f8e1-427d-9440-afdff59bf728 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:41.982490 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64abfb2c-f8e1-427d-9440-afdff59bf728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:41.985171 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64abfb2c-f8e1-427d-9440-afdff59bf728 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:41.986614 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64abfb2c-f8e1-427d-9440-afdff59bf728 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:41.991348 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64abfb2c-f8e1-427d-9440-afdff59bf728 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:42.077998 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 980/1927] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:30:41 2022] POST /identity/v3/projects => generated 383 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:30:42.086365 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64abfb2c-f8e1-427d-9440-afdff59bf728 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:42.165132 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23cd0cc-d79b-43a7-a9a1-2de741c83d18 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:42.306626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64abfb2c-f8e1-427d-9440-afdff59bf728 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:42.315977 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a81f0-a3e1-42d5-9e21-1464f538af80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:42.319555 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a81f0-a3e1-42d5-9e21-1464f538af80 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:42.320248 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a81f0-a3e1-42d5-9e21-1464f538af80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:42.320945 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a81f0-a3e1-42d5-9e21-1464f538af80 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:42.322346 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84a81f0-a3e1-42d5-9e21-1464f538af80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:42.328483 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84a81f0-a3e1-42d5-9e21-1464f538af80 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:42.349160 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ad6fc-086d-4d27-a29b-72a1d0e5f163 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:42.350395 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ad6fc-086d-4d27-a29b-72a1d0e5f163 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:42.350870 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ad6fc-086d-4d27-a29b-72a1d0e5f163 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:42.351350 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ad6fc-086d-4d27-a29b-72a1d0e5f163 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:42.383151 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 981/1928] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:30:42 2022] POST /identity/v3/users => generated 298 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:30:42.396493 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366ad6fc-086d-4d27-a29b-72a1d0e5f163 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:42.396798 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a81f0-a3e1-42d5-9e21-1464f538af80 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:42.400036 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e84a81f0-a3e1-42d5-9e21-1464f538af80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:42.404274 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366ad6fc-086d-4d27-a29b-72a1d0e5f163 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:42.425911 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a81f0-a3e1-42d5-9e21-1464f538af80 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:42.437408 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f8fe1-3dcd-4206-81aa-53479f309c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:42.444218 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f8fe1-3dcd-4206-81aa-53479f309c05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:42.444709 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f8fe1-3dcd-4206-81aa-53479f309c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:42.445190 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f8fe1-3dcd-4206-81aa-53479f309c05 admin admin] PATH_INFO: `/v3/projects/681f57f09ffe48139ab8a13d82c75e7c` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:42.447990 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4f8fe1-3dcd-4206-81aa-53479f309c05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=681f57f09ffe48139ab8a13d82c75e7c)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:42.452156 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4f8fe1-3dcd-4206-81aa-53479f309c05 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:42.463159 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 948/1929] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:30:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:30:42.464566 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ad6fc-086d-4d27-a29b-72a1d0e5f163 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:42.500214 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ad6fc-086d-4d27-a29b-72a1d0e5f163 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:42.521588 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c571695-c886-43d2-b2c0-e0b2570074a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:42.522179 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c571695-c886-43d2-b2c0-e0b2570074a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:42.524006 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c571695-c886-43d2-b2c0-e0b2570074a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:42.524631 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c571695-c886-43d2-b2c0-e0b2570074a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:42.526421 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c571695-c886-43d2-b2c0-e0b2570074a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:42.530492 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c571695-c886-43d2-b2c0-e0b2570074a9 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:42.562963 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 949/1930] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:42 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:30:42.564583 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c571695-c886-43d2-b2c0-e0b2570074a9 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:42.585951 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c571695-c886-43d2-b2c0-e0b2570074a9 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:42.591450 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-6c4f8fe1-3dcd-4206-81aa-53479f309c05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '681f57f09ffe48139ab8a13d82c75e7c', 'request_id': 'req-6c4f8fe1-3dcd-4206-81aa-53479f309c05'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:42.611223 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-6c4f8fe1-3dcd-4206-81aa-53479f309c05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '681f57f09ffe48139ab8a13d82c75e7c', 'request_id': 'req-6c4f8fe1-3dcd-4206-81aa-53479f309c05'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:42.654181 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-6c4f8fe1-3dcd-4206-81aa-53479f309c05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '681f57f09ffe48139ab8a13d82c75e7c', 'request_id': 'req-6c4f8fe1-3dcd-4206-81aa-53479f309c05'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:42.686298 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-6c4f8fe1-3dcd-4206-81aa-53479f309c05 admin admin] The token cache is being invalidate because project 681f57f09ffe48139ab8a13d82c75e7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:30:42.687111 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-6c4f8fe1-3dcd-4206-81aa-53479f309c05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:42.690036 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-6c4f8fe1-3dcd-4206-81aa-53479f309c05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:42.701732 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 982/1931] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:30:42 2022] DELETE /identity/v3/projects/681f57f09ffe48139ab8a13d82c75e7c => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:42.714437 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651957b3-dd3c-4ce3-b76e-a0c416f0593d None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:42.715411 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651957b3-dd3c-4ce3-b76e-a0c416f0593d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:42.716283 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651957b3-dd3c-4ce3-b76e-a0c416f0593d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:42.808086 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-651957b3-dd3c-4ce3-b76e-a0c416f0593d None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:42.816709 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-651957b3-dd3c-4ce3-b76e-a0c416f0593d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:42.846429 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0596ef0f-912c-469a-ab46-5bff84dd3bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:42.868240 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0596ef0f-912c-469a-ab46-5bff84dd3bf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:42.868837 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0596ef0f-912c-469a-ab46-5bff84dd3bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:42.869240 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0596ef0f-912c-469a-ab46-5bff84dd3bf0 admin admin] PATH_INFO: `/v3/projects/73fcc61fd03d4831a1d5e137e4b229b7/users/d9c32b0a455c4382b2ff86d023238077/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:42.916757 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0596ef0f-912c-469a-ab46-5bff84dd3bf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73fcc61fd03d4831a1d5e137e4b229b7, user_id=d9c32b0a455c4382b2ff86d023238077, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:42.928683 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0596ef0f-912c-469a-ab46-5bff84dd3bf0 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:42.946259 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:42.974045 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:43.010534 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 950/1932] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:30:42 2022] PUT /identity/v3/projects/73fcc61fd03d4831a1d5e137e4b229b7/users/d9c32b0a455c4382b2ff86d023238077/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:43.017032 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0596ef0f-912c-469a-ab46-5bff84dd3bf0 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:43.024710 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0596ef0f-912c-469a-ab46-5bff84dd3bf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:43.061466 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 983/1933] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:43.080188 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4970c1-e54f-409c-9708-da1f5e0030aa None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:43.080751 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4970c1-e54f-409c-9708-da1f5e0030aa None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:43.081301 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4970c1-e54f-409c-9708-da1f5e0030aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:43.194214 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-5a4970c1-e54f-409c-9708-da1f5e0030aa None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:43.195816 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5a4970c1-e54f-409c-9708-da1f5e0030aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:43.306863 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0596ef0f-912c-469a-ab46-5bff84dd3bf0 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:43.325037 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb7a80-f95d-443a-a73a-a55c676bb080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:43.329536 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb7a80-f95d-443a-a73a-a55c676bb080 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:43.329974 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb7a80-f95d-443a-a73a-a55c676bb080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:43.330314 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb7a80-f95d-443a-a73a-a55c676bb080 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:43.331747 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fb7a80-f95d-443a-a73a-a55c676bb080 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:43.336696 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fb7a80-f95d-443a-a73a-a55c676bb080 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:43.401724 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 951/1934] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:30:43.402964 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb7a80-f95d-443a-a73a-a55c676bb080 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:43.450314 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb7a80-f95d-443a-a73a-a55c676bb080 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:43.489975 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94ddcb0-c1d3-4ec2-8e1c-15fb2afaa447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:43.491257 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ddcb0-c1d3-4ec2-8e1c-15fb2afaa447 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:43.499821 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ddcb0-c1d3-4ec2-8e1c-15fb2afaa447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:43.500370 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ddcb0-c1d3-4ec2-8e1c-15fb2afaa447 admin admin] PATH_INFO: `/v3/projects/73fcc61fd03d4831a1d5e137e4b229b7/users/d9c32b0a455c4382b2ff86d023238077/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:43.510590 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 984/1935] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 433 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:43.512869 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94ddcb0-c1d3-4ec2-8e1c-15fb2afaa447 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73fcc61fd03d4831a1d5e137e4b229b7, user_id=d9c32b0a455c4382b2ff86d023238077, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:43.525476 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4970c1-e54f-409c-9708-da1f5e0030aa None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:43.526674 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94ddcb0-c1d3-4ec2-8e1c-15fb2afaa447 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:43.569598 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4970c1-e54f-409c-9708-da1f5e0030aa None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:43.573013 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 952/1936] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:30:43 2022] PUT /identity/v3/projects/73fcc61fd03d4831a1d5e137e4b229b7/users/d9c32b0a455c4382b2ff86d023238077/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:43.577435 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94ddcb0-c1d3-4ec2-8e1c-15fb2afaa447 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:43.595312 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a94ddcb0-c1d3-4ec2-8e1c-15fb2afaa447 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:43.852104 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec3aa36-967d-483e-b45d-795186293416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:43.853335 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec3aa36-967d-483e-b45d-795186293416 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:43.854752 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec3aa36-967d-483e-b45d-795186293416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:43.855288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec3aa36-967d-483e-b45d-795186293416 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:43.857146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec3aa36-967d-483e-b45d-795186293416 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:30:43.857798 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec3aa36-967d-483e-b45d-795186293416 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:43.861479 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec3aa36-967d-483e-b45d-795186293416 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:43.934790 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 985/1937] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:30:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:30:43.944720 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb4e23-b52c-4cef-bb9e-fa33292b700d None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:43.946490 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb4e23-b52c-4cef-bb9e-fa33292b700d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:43.946490 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb4e23-b52c-4cef-bb9e-fa33292b700d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:43.960205 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94ddcb0-c1d3-4ec2-8e1c-15fb2afaa447 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:43.969770 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bd131c-c332-41d6-9062-96308ef4868a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:43.973344 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd131c-c332-41d6-9062-96308ef4868a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:43.974253 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd131c-c332-41d6-9062-96308ef4868a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:43.974974 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd131c-c332-41d6-9062-96308ef4868a admin admin] PATH_INFO: `/v3/users/62fa006ccf304457abe457dd688386ce` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:43.977827 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bd131c-c332-41d6-9062-96308ef4868a admin admin] RBAC: Authorizing `identity:delete_user(user_id=62fa006ccf304457abe457dd688386ce)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:43.983745 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bd131c-c332-41d6-9062-96308ef4868a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:44.039435 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-1cdb4e23-b52c-4cef-bb9e-fa33292b700d None None] MFA Rules not processed for user `d9c32b0a455c4382b2ff86d023238077`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:44.047051 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-1cdb4e23-b52c-4cef-bb9e-fa33292b700d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:44.174434 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-f5bd131c-c332-41d6-9062-96308ef4868a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62fa006ccf304457abe457dd688386ce'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:44.209931 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-f5bd131c-c332-41d6-9062-96308ef4868a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62fa006ccf304457abe457dd688386ce'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:44.212300 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-f5bd131c-c332-41d6-9062-96308ef4868a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62fa006ccf304457abe457dd688386ce'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:44.217303 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-f5bd131c-c332-41d6-9062-96308ef4868a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62fa006ccf304457abe457dd688386ce'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:44.269785 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-f5bd131c-c332-41d6-9062-96308ef4868a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62fa006ccf304457abe457dd688386ce'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:44.297004 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:44.303460 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:44.339540 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 986/1938] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:43 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:44.342180 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb4e23-b52c-4cef-bb9e-fa33292b700d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:44.345089 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-1cdb4e23-b52c-4cef-bb9e-fa33292b700d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:44.359895 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 953/1939] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:30:43 2022] DELETE /identity/v3/users/62fa006ccf304457abe457dd688386ce => generated 0 bytes in 778 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:44.364815 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bd131c-c332-41d6-9062-96308ef4868a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:44.370815 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f5bd131c-c332-41d6-9062-96308ef4868a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:44.387129 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb4e23-b52c-4cef-bb9e-fa33292b700d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:44.637402 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd18878a-833c-4c38-8945-0bbb547a8d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:44.637402 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18878a-833c-4c38-8945-0bbb547a8d71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:44.637402 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18878a-833c-4c38-8945-0bbb547a8d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:44.637996 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18878a-833c-4c38-8945-0bbb547a8d71 admin admin] PATH_INFO: `/v3/projects/9dc4b89703074c6f9709c99a45a94f29` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:44.641934 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd18878a-833c-4c38-8945-0bbb547a8d71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9dc4b89703074c6f9709c99a45a94f29)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:44.646150 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd18878a-833c-4c38-8945-0bbb547a8d71 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:44.709264 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bd131c-c332-41d6-9062-96308ef4868a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:44.723845 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d7b9b-12b9-4696-b6d1-99938f3d584e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:44.726407 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d7b9b-12b9-4696-b6d1-99938f3d584e admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:44.728766 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d7b9b-12b9-4696-b6d1-99938f3d584e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:44.729504 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d7b9b-12b9-4696-b6d1-99938f3d584e admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:44.732077 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d7b9b-12b9-4696-b6d1-99938f3d584e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:44.741086 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d7b9b-12b9-4696-b6d1-99938f3d584e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:44.797004 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-bd18878a-833c-4c38-8945-0bbb547a8d71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9dc4b89703074c6f9709c99a45a94f29', 'request_id': 'req-bd18878a-833c-4c38-8945-0bbb547a8d71'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:44.820572 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 954/1940] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:30:44 2022] POST /identity/v3/projects => generated 373 bytes in 454 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:30:44.820572 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef4ca05-65a3-46e1-82d2-3f51e8eb89f5 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:44.825906 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef4ca05-65a3-46e1-82d2-3f51e8eb89f5 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:44.826563 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef4ca05-65a3-46e1-82d2-3f51e8eb89f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:44.832612 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-bd18878a-833c-4c38-8945-0bbb547a8d71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9dc4b89703074c6f9709c99a45a94f29', 'request_id': 'req-bd18878a-833c-4c38-8945-0bbb547a8d71'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:44.864232 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-bd18878a-833c-4c38-8945-0bbb547a8d71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9dc4b89703074c6f9709c99a45a94f29', 'request_id': 'req-bd18878a-833c-4c38-8945-0bbb547a8d71'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:44.900546 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-9ef4ca05-65a3-46e1-82d2-3f51e8eb89f5 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:44.907216 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-bd18878a-833c-4c38-8945-0bbb547a8d71 admin admin] The token cache is being invalidate because project 9dc4b89703074c6f9709c99a45a94f29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:30:44.907504 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-bd18878a-833c-4c38-8945-0bbb547a8d71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:44.908868 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9ef4ca05-65a3-46e1-82d2-3f51e8eb89f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:44.911508 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-bd18878a-833c-4c38-8945-0bbb547a8d71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:44.918769 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 987/1941] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:30:44 2022] DELETE /identity/v3/projects/9dc4b89703074c6f9709c99a45a94f29 => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:44.925215 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd18878a-833c-4c38-8945-0bbb547a8d71 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:44.932557 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-bd18878a-833c-4c38-8945-0bbb547a8d71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:45.098859 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 955/1942] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:45.100464 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef4ca05-65a3-46e1-82d2-3f51e8eb89f5 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:45.103650 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9ef4ca05-65a3-46e1-82d2-3f51e8eb89f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:45.126735 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef4ca05-65a3-46e1-82d2-3f51e8eb89f5 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:45.137822 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169c7178-73aa-488d-8ab5-3607d297921e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:45.139080 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c7178-73aa-488d-8ab5-3607d297921e admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:45.139548 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c7178-73aa-488d-8ab5-3607d297921e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:45.139912 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c7178-73aa-488d-8ab5-3607d297921e admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:45.141614 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169c7178-73aa-488d-8ab5-3607d297921e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:45.148282 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169c7178-73aa-488d-8ab5-3607d297921e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:45.164428 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd18878a-833c-4c38-8945-0bbb547a8d71 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:45.204033 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 956/1943] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:30:45 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:30:45.205408 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169c7178-73aa-488d-8ab5-3607d297921e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:45.208402 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-169c7178-73aa-488d-8ab5-3607d297921e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:45.217900 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:45.235280 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169c7178-73aa-488d-8ab5-3607d297921e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:45.264933 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9455e744-ac6f-49d6-ac1e-3bce3b94c91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:45.267370 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455e744-ac6f-49d6-ac1e-3bce3b94c91e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:45.268257 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455e744-ac6f-49d6-ac1e-3bce3b94c91e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:45.268664 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455e744-ac6f-49d6-ac1e-3bce3b94c91e admin admin] PATH_INFO: `/v3/users/92454b671c434427857c73e135f2fd09` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:45.271187 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9455e744-ac6f-49d6-ac1e-3bce3b94c91e admin admin] RBAC: Authorizing `identity:delete_user(user_id=92454b671c434427857c73e135f2fd09)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:45.278788 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9455e744-ac6f-49d6-ac1e-3bce3b94c91e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:45.315301 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc012f05-f63e-4a2a-a706-a3a84e40ddee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:45.316873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc012f05-f63e-4a2a-a706-a3a84e40ddee service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:45.317682 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc012f05-f63e-4a2a-a706-a3a84e40ddee service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:45.319632 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc012f05-f63e-4a2a-a706-a3a84e40ddee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:45.326656 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-dc012f05-f63e-4a2a-a706-a3a84e40ddee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:45.407039 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9455e744-ac6f-49d6-ac1e-3bce3b94c91e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92454b671c434427857c73e135f2fd09'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:45.426967 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9455e744-ac6f-49d6-ac1e-3bce3b94c91e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92454b671c434427857c73e135f2fd09'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:45.429978 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9455e744-ac6f-49d6-ac1e-3bce3b94c91e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92454b671c434427857c73e135f2fd09'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:45.431099 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9455e744-ac6f-49d6-ac1e-3bce3b94c91e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92454b671c434427857c73e135f2fd09'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:45.446661 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:45.474221 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9455e744-ac6f-49d6-ac1e-3bce3b94c91e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92454b671c434427857c73e135f2fd09'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:45.502727 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 957/1944] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:30:45 2022] DELETE /identity/v3/users/92454b671c434427857c73e135f2fd09 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:45.504531 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9455e744-ac6f-49d6-ac1e-3bce3b94c91e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:45.506258 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9455e744-ac6f-49d6-ac1e-3bce3b94c91e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:45.586809 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc012f05-f63e-4a2a-a706-a3a84e40ddee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:45.588408 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc012f05-f63e-4a2a-a706-a3a84e40ddee service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:45.621441 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 988/1945] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:30:44 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 702 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:30:45.622837 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc012f05-f63e-4a2a-a706-a3a84e40ddee service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:45.658613 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc012f05-f63e-4a2a-a706-a3a84e40ddee service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:45.793246 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9455e744-ac6f-49d6-ac1e-3bce3b94c91e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:45.813214 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8838cd75-04d6-4494-a3a5-d6dc00a87b8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:45.817211 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838cd75-04d6-4494-a3a5-d6dc00a87b8f service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:45.820780 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838cd75-04d6-4494-a3a5-d6dc00a87b8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:45.821326 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838cd75-04d6-4494-a3a5-d6dc00a87b8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:45.829353 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8838cd75-04d6-4494-a3a5-d6dc00a87b8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:45.953418 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc85f55-5d19-431e-9c00-10da39ab5160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:45.997103 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc85f55-5d19-431e-9c00-10da39ab5160 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:45.999003 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc85f55-5d19-431e-9c00-10da39ab5160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:45.999919 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc85f55-5d19-431e-9c00-10da39ab5160 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:46.005679 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc85f55-5d19-431e-9c00-10da39ab5160 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:46.012267 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc85f55-5d19-431e-9c00-10da39ab5160 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:46.080191 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 989/1946] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 458 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:30:46.089282 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc85f55-5d19-431e-9c00-10da39ab5160 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:46.095366 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-1dc85f55-5d19-431e-9c00-10da39ab5160 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:46.160266 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc85f55-5d19-431e-9c00-10da39ab5160 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:46.186291 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8838cd75-04d6-4494-a3a5-d6dc00a87b8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:46.188317 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8838cd75-04d6-4494-a3a5-d6dc00a87b8f service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:46.190184 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d29a12-8085-4cbe-9487-4a190ec188ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:46.196469 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d29a12-8085-4cbe-9487-4a190ec188ce admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:46.201083 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d29a12-8085-4cbe-9487-4a190ec188ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:46.202047 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d29a12-8085-4cbe-9487-4a190ec188ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:46.207792 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d29a12-8085-4cbe-9487-4a190ec188ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:46.216696 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 958/1947] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:30:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 713 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:30:46.219623 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d29a12-8085-4cbe-9487-4a190ec188ce admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:46.285980 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 990/1948] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:30:46.290303 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8838cd75-04d6-4494-a3a5-d6dc00a87b8f service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:46.316262 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8838cd75-04d6-4494-a3a5-d6dc00a87b8f service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:46.338569 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0086e1fd-c319-484a-a57a-20346d08806b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:46.342463 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0086e1fd-c319-484a-a57a-20346d08806b admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:46.343594 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0086e1fd-c319-484a-a57a-20346d08806b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:46.346188 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0086e1fd-c319-484a-a57a-20346d08806b admin admin] PATH_INFO: `/v3/projects/0307481a33be4b268d2793bb65f9d9ef/users/82c323bc7a8c45e39714bb3e7f401c3a/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:46.453691 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0086e1fd-c319-484a-a57a-20346d08806b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0307481a33be4b268d2793bb65f9d9ef, user_id=82c323bc7a8c45e39714bb3e7f401c3a, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:46.473159 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0086e1fd-c319-484a-a57a-20346d08806b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:46.488362 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d29a12-8085-4cbe-9487-4a190ec188ce admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:46.491101 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e9d29a12-8085-4cbe-9487-4a190ec188ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:46.558439 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d29a12-8085-4cbe-9487-4a190ec188ce admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:46.640332 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 959/1949] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:30:46 2022] PUT /identity/v3/projects/0307481a33be4b268d2793bb65f9d9ef/users/82c323bc7a8c45e39714bb3e7f401c3a/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:46.654663 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251764bf-5832-4dd5-980d-afb0e00e432d None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:46.656652 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251764bf-5832-4dd5-980d-afb0e00e432d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:46.658117 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251764bf-5832-4dd5-980d-afb0e00e432d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:46.739058 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-251764bf-5832-4dd5-980d-afb0e00e432d None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:46.740387 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-251764bf-5832-4dd5-980d-afb0e00e432d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:46.881354 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:46.917893 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747bffa8-d9f5-4073-9895-17f1487bcebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:46.924155 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747bffa8-d9f5-4073-9895-17f1487bcebe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:46.924789 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747bffa8-d9f5-4073-9895-17f1487bcebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:46.925243 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747bffa8-d9f5-4073-9895-17f1487bcebe admin admin] PATH_INFO: `/v3/projects/0211aa0e968c475a936d67fe00052b21` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:46.932775 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747bffa8-d9f5-4073-9895-17f1487bcebe admin admin] RBAC: Authorizing `identity:delete_project(project_id=0211aa0e968c475a936d67fe00052b21)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:46.942613 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747bffa8-d9f5-4073-9895-17f1487bcebe admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:47.100984 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 960/1950] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 449 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:47.134012 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251764bf-5832-4dd5-980d-afb0e00e432d None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:47.136978 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-251764bf-5832-4dd5-980d-afb0e00e432d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:47.214260 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:47.270839 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-747bffa8-d9f5-4073-9895-17f1487bcebe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0211aa0e968c475a936d67fe00052b21', 'request_id': 'req-747bffa8-d9f5-4073-9895-17f1487bcebe'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:47.304056 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-747bffa8-d9f5-4073-9895-17f1487bcebe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0211aa0e968c475a936d67fe00052b21', 'request_id': 'req-747bffa8-d9f5-4073-9895-17f1487bcebe'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:47.331420 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-747bffa8-d9f5-4073-9895-17f1487bcebe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0211aa0e968c475a936d67fe00052b21', 'request_id': 'req-747bffa8-d9f5-4073-9895-17f1487bcebe'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:47.369371 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-747bffa8-d9f5-4073-9895-17f1487bcebe admin admin] The token cache is being invalidate because project 0211aa0e968c475a936d67fe00052b21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:30:47.369839 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-747bffa8-d9f5-4073-9895-17f1487bcebe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:47.370931 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-747bffa8-d9f5-4073-9895-17f1487bcebe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:47.374893 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 991/1951] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:30:46 2022] DELETE /identity/v3/projects/0211aa0e968c475a936d67fe00052b21 => generated 0 bytes in 887 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:30:47.386353 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc3fda-3ff4-40d5-8946-559987a9cd14 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:47.386897 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc3fda-3ff4-40d5-8946-559987a9cd14 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:47.387437 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc3fda-3ff4-40d5-8946-559987a9cd14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:47.426500 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251764bf-5832-4dd5-980d-afb0e00e432d None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:47.450348 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-30cc3fda-3ff4-40d5-8946-559987a9cd14 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:47.457817 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-30cc3fda-3ff4-40d5-8946-559987a9cd14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:47.600049 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c614d6-f9e1-4e91-8ddd-9f1f18eea97b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:47.602807 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c614d6-f9e1-4e91-8ddd-9f1f18eea97b service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:47.603505 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c614d6-f9e1-4e91-8ddd-9f1f18eea97b service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:47.604056 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c614d6-f9e1-4e91-8ddd-9f1f18eea97b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:47.609193 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a8c614d6-f9e1-4e91-8ddd-9f1f18eea97b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:47.640426 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 992/1952] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:30:47.640426 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cc3fda-3ff4-40d5-8946-559987a9cd14 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:47.640426 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-30cc3fda-3ff4-40d5-8946-559987a9cd14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:47.658875 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c614d6-f9e1-4e91-8ddd-9f1f18eea97b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:47.661891 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c614d6-f9e1-4e91-8ddd-9f1f18eea97b service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:47.664699 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cc3fda-3ff4-40d5-8946-559987a9cd14 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:47.673307 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df6d8c6-6da6-45d8-997c-5506216071b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:47.675388 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df6d8c6-6da6-45d8-997c-5506216071b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:47.675889 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df6d8c6-6da6-45d8-997c-5506216071b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:47.677317 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df6d8c6-6da6-45d8-997c-5506216071b3 admin admin] PATH_INFO: `/v3/projects/84f9e28ba9774ba4af4e255b84ecf74c` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:47.680063 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df6d8c6-6da6-45d8-997c-5506216071b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84f9e28ba9774ba4af4e255b84ecf74c)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:47.683272 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 961/1953] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:30:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:30:47.683645 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df6d8c6-6da6-45d8-997c-5506216071b3 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:47.686806 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e24b82-c937-4a84-92e7-f7a2f21f8c56 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:47.687465 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e24b82-c937-4a84-92e7-f7a2f21f8c56 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:47.688892 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e24b82-c937-4a84-92e7-f7a2f21f8c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:47.734976 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-74e24b82-c937-4a84-92e7-f7a2f21f8c56 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:47.737073 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-74e24b82-c937-4a84-92e7-f7a2f21f8c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:47.756047 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 962/1954] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:47.762520 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e24b82-c937-4a84-92e7-f7a2f21f8c56 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:47.764705 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-74e24b82-c937-4a84-92e7-f7a2f21f8c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:47.839340 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7df6d8c6-6da6-45d8-997c-5506216071b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84f9e28ba9774ba4af4e255b84ecf74c', 'request_id': 'req-7df6d8c6-6da6-45d8-997c-5506216071b3'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:47.867109 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7df6d8c6-6da6-45d8-997c-5506216071b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84f9e28ba9774ba4af4e255b84ecf74c', 'request_id': 'req-7df6d8c6-6da6-45d8-997c-5506216071b3'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:47.888638 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7df6d8c6-6da6-45d8-997c-5506216071b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84f9e28ba9774ba4af4e255b84ecf74c', 'request_id': 'req-7df6d8c6-6da6-45d8-997c-5506216071b3'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:47.916513 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7df6d8c6-6da6-45d8-997c-5506216071b3 admin admin] The token cache is being invalidate because project 84f9e28ba9774ba4af4e255b84ecf74c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:30:47.917337 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7df6d8c6-6da6-45d8-997c-5506216071b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:47.918669 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7df6d8c6-6da6-45d8-997c-5506216071b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:47.921590 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 993/1955] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:30:47 2022] DELETE /identity/v3/projects/84f9e28ba9774ba4af4e255b84ecf74c => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:47.922823 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df6d8c6-6da6-45d8-997c-5506216071b3 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:47.925126 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7df6d8c6-6da6-45d8-997c-5506216071b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:47.931766 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e24b82-c937-4a84-92e7-f7a2f21f8c56 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:47.952274 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:48.053704 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc5209-ce51-4fd1-bbac-316f1d7687ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:48.054997 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc5209-ce51-4fd1-bbac-316f1d7687ad service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:48.055527 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc5209-ce51-4fd1-bbac-316f1d7687ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:48.056044 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc5209-ce51-4fd1-bbac-316f1d7687ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:48.059642 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-87bc5209-ce51-4fd1-bbac-316f1d7687ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:48.129267 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df6d8c6-6da6-45d8-997c-5506216071b3 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:48.132833 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d22d33c-2ccd-4210-aff1-0a09e0c4f88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:48.133619 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22d33c-2ccd-4210-aff1-0a09e0c4f88a admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:48.133998 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22d33c-2ccd-4210-aff1-0a09e0c4f88a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:48.134295 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22d33c-2ccd-4210-aff1-0a09e0c4f88a admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:48.135161 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d22d33c-2ccd-4210-aff1-0a09e0c4f88a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:30:48.135526 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d22d33c-2ccd-4210-aff1-0a09e0c4f88a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:48.137289 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d22d33c-2ccd-4210-aff1-0a09e0c4f88a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:48.152650 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:48.165478 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 994/1956] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:30:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:30:48.168574 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d22d33c-2ccd-4210-aff1-0a09e0c4f88a admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:48.175933 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3d22d33c-2ccd-4210-aff1-0a09e0c4f88a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:48.222819 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d22d33c-2ccd-4210-aff1-0a09e0c4f88a admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:48.234063 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b95880-2bd8-4cc0-b4cf-df18e36211c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:48.235871 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b95880-2bd8-4cc0-b4cf-df18e36211c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:48.236660 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b95880-2bd8-4cc0-b4cf-df18e36211c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:48.236660 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b95880-2bd8-4cc0-b4cf-df18e36211c8 admin admin] PATH_INFO: `/v3/users/c7ce78c48f394382a0493e7ea6d1971b` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:48.239892 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b95880-2bd8-4cc0-b4cf-df18e36211c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7ce78c48f394382a0493e7ea6d1971b)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:48.243959 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b95880-2bd8-4cc0-b4cf-df18e36211c8 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:48.263248 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bc5209-ce51-4fd1-bbac-316f1d7687ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:48.274018 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bc5209-ce51-4fd1-bbac-316f1d7687ad service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:48.313784 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 963/1957] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:30:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:30:48.315257 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc5209-ce51-4fd1-bbac-316f1d7687ad service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:48.354882 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc5209-ce51-4fd1-bbac-316f1d7687ad service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:48.367377 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-99b95880-2bd8-4cc0-b4cf-df18e36211c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7ce78c48f394382a0493e7ea6d1971b'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:48.380929 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a031a865-d81d-4355-98f3-233521f140fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:48.382035 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031a865-d81d-4355-98f3-233521f140fb admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:48.382477 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031a865-d81d-4355-98f3-233521f140fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:48.383079 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031a865-d81d-4355-98f3-233521f140fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:48.384236 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a031a865-d81d-4355-98f3-233521f140fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:48.393535 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a031a865-d81d-4355-98f3-233521f140fb admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:48.416634 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-99b95880-2bd8-4cc0-b4cf-df18e36211c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7ce78c48f394382a0493e7ea6d1971b'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:48.418373 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-99b95880-2bd8-4cc0-b4cf-df18e36211c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7ce78c48f394382a0493e7ea6d1971b'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:48.428707 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-99b95880-2bd8-4cc0-b4cf-df18e36211c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7ce78c48f394382a0493e7ea6d1971b'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:48.458248 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-99b95880-2bd8-4cc0-b4cf-df18e36211c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7ce78c48f394382a0493e7ea6d1971b'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:48.509132 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 964/1958] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:30:48 2022] POST /identity/v3/projects => generated 359 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:30:48.510531 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a031a865-d81d-4355-98f3-233521f140fb admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:48.513445 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a031a865-d81d-4355-98f3-233521f140fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:48.549391 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 995/1959] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:30:48 2022] DELETE /identity/v3/users/c7ce78c48f394382a0493e7ea6d1971b => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:48.604766 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c413739-d21c-4855-9317-78d95e471be6 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:48.605336 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c413739-d21c-4855-9317-78d95e471be6 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:48.607661 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c413739-d21c-4855-9317-78d95e471be6 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:48.610829 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 996/1960] 10.222.0.87 () {54 vars in 921 bytes} [Tue Jun 21 18:30:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:30:48.637010 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427041b9-c58d-43b7-a92d-1b59a4d4df58 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:48.637557 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427041b9-c58d-43b7-a92d-1b59a4d4df58 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:48.638325 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427041b9-c58d-43b7-a92d-1b59a4d4df58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:48.875025 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a031a865-d81d-4355-98f3-233521f140fb admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:49.127457 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-427041b9-c58d-43b7-a92d-1b59a4d4df58 None None] MFA Rules not processed for user `a1b3fa67905c4027b3656ac36e2b76c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:49.130060 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-427041b9-c58d-43b7-a92d-1b59a4d4df58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:49.179668 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c94bbe-5a37-42e4-a8ad-c99b0675e444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:49.180650 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c94bbe-5a37-42e4-a8ad-c99b0675e444 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:49.183538 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c94bbe-5a37-42e4-a8ad-c99b0675e444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:49.184058 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c94bbe-5a37-42e4-a8ad-c99b0675e444 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:49.185325 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c94bbe-5a37-42e4-a8ad-c99b0675e444 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:49.189524 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c94bbe-5a37-42e4-a8ad-c99b0675e444 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:49.271618 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 965/1961] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:30:48 2022] POST /identity/v3/users => generated 280 bytes in 762 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:30:49.282047 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c94bbe-5a37-42e4-a8ad-c99b0675e444 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:49.326008 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:49.339772 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c94bbe-5a37-42e4-a8ad-c99b0675e444 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:49.367484 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a16905-1a77-46a5-b35e-5df898b6034b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:49.373764 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a16905-1a77-46a5-b35e-5df898b6034b admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:49.375871 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a16905-1a77-46a5-b35e-5df898b6034b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:49.379746 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a16905-1a77-46a5-b35e-5df898b6034b admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:49.394292 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a16905-1a77-46a5-b35e-5df898b6034b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:49.405156 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a16905-1a77-46a5-b35e-5df898b6034b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:49.500038 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 966/1962] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:30:49.508958 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a16905-1a77-46a5-b35e-5df898b6034b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:49.574108 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a16905-1a77-46a5-b35e-5df898b6034b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:49.602888 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4337c39-208c-40be-a67b-ef9c16d7b380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:49.604766 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 997/1963] 10.222.0.87 () {58 vars in 1018 bytes} [Tue Jun 21 18:30:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 971 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:49.610007 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4337c39-208c-40be-a67b-ef9c16d7b380 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:49.611311 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4337c39-208c-40be-a67b-ef9c16d7b380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:49.613224 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4337c39-208c-40be-a67b-ef9c16d7b380 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:49.621140 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4337c39-208c-40be-a67b-ef9c16d7b380 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:49.623968 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4337c39-208c-40be-a67b-ef9c16d7b380 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:49.664689 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427041b9-c58d-43b7-a92d-1b59a4d4df58 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:49.668220 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-427041b9-c58d-43b7-a92d-1b59a4d4df58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:49.685770 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 967/1964] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:30:49.687098 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4337c39-208c-40be-a67b-ef9c16d7b380 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:49.726246 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4337c39-208c-40be-a67b-ef9c16d7b380 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:49.740682 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427041b9-c58d-43b7-a92d-1b59a4d4df58 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:49.746863 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a9628c-d5d2-42b4-94a0-228c878ac3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:49.748385 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9628c-d5d2-42b4-94a0-228c878ac3f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:49.748764 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9628c-d5d2-42b4-94a0-228c878ac3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:49.749302 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9628c-d5d2-42b4-94a0-228c878ac3f9 admin admin] PATH_INFO: `/v3/projects/6dfc97ee2ef947bd868e00767d3ece77/users/fc5d94367276402b97ad8d844ffbd5de/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:49.774084 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5fc19c-79d0-4a6e-87e4-e67d9eef965e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:49.778494 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5fc19c-79d0-4a6e-87e4-e67d9eef965e service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:49.784096 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5fc19c-79d0-4a6e-87e4-e67d9eef965e service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:49.786035 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5fc19c-79d0-4a6e-87e4-e67d9eef965e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:49.825594 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a9628c-d5d2-42b4-94a0-228c878ac3f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dfc97ee2ef947bd868e00767d3ece77, user_id=fc5d94367276402b97ad8d844ffbd5de, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:49.831889 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a9628c-d5d2-42b4-94a0-228c878ac3f9 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:49.864075 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:49.872128 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5fc19c-79d0-4a6e-87e4-e67d9eef965e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:49.875094 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5fc19c-79d0-4a6e-87e4-e67d9eef965e service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:49.892305 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 968/1965] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:30:49 2022] PUT /identity/v3/projects/6dfc97ee2ef947bd868e00767d3ece77/users/fc5d94367276402b97ad8d844ffbd5de/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:49.915809 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a9628c-d5d2-42b4-94a0-228c878ac3f9 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:49.925691 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a3a9628c-d5d2-42b4-94a0-228c878ac3f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:49.973281 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 998/1966] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:30:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:30:49.988588 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c3318-ff84-487c-b9d9-922829208bb9 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:49.991054 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c3318-ff84-487c-b9d9-922829208bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:49.999169 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c3318-ff84-487c-b9d9-922829208bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:50.158263 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-6b0c3318-ff84-487c-b9d9-922829208bb9 None None] MFA Rules not processed for user `fc5d94367276402b97ad8d844ffbd5de`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:50.160510 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-6b0c3318-ff84-487c-b9d9-922829208bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:50.206972 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a9628c-d5d2-42b4-94a0-228c878ac3f9 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:50.234305 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164734e7-7a65-4b19-ba9e-8bbcbdf4be35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:50.239030 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164734e7-7a65-4b19-ba9e-8bbcbdf4be35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:50.241357 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164734e7-7a65-4b19-ba9e-8bbcbdf4be35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:50.242245 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164734e7-7a65-4b19-ba9e-8bbcbdf4be35 admin admin] PATH_INFO: `/v3/projects/a399367587c2486f86f2039e5c3ebd58` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:50.248787 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164734e7-7a65-4b19-ba9e-8bbcbdf4be35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a399367587c2486f86f2039e5c3ebd58)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:50.257729 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164734e7-7a65-4b19-ba9e-8bbcbdf4be35 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:50.531583 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-164734e7-7a65-4b19-ba9e-8bbcbdf4be35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a399367587c2486f86f2039e5c3ebd58', 'request_id': 'req-164734e7-7a65-4b19-ba9e-8bbcbdf4be35'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:50.537715 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 999/1967] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:49 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 560 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:50.566057 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-164734e7-7a65-4b19-ba9e-8bbcbdf4be35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a399367587c2486f86f2039e5c3ebd58', 'request_id': 'req-164734e7-7a65-4b19-ba9e-8bbcbdf4be35'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:50.588539 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-164734e7-7a65-4b19-ba9e-8bbcbdf4be35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a399367587c2486f86f2039e5c3ebd58', 'request_id': 'req-164734e7-7a65-4b19-ba9e-8bbcbdf4be35'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:50.606061 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c3318-ff84-487c-b9d9-922829208bb9 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:50.609108 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-6b0c3318-ff84-487c-b9d9-922829208bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:50.630792 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-164734e7-7a65-4b19-ba9e-8bbcbdf4be35 admin admin] The token cache is being invalidate because project a399367587c2486f86f2039e5c3ebd58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:30:50.631206 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-164734e7-7a65-4b19-ba9e-8bbcbdf4be35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:50.636556 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-164734e7-7a65-4b19-ba9e-8bbcbdf4be35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:50.662780 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 969/1968] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:30:49 2022] DELETE /identity/v3/projects/a399367587c2486f86f2039e5c3ebd58 => generated 0 bytes in 770 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:50.686310 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562a7d3-a817-482c-8100-b96b9867fa0d None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:50.687092 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562a7d3-a817-482c-8100-b96b9867fa0d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:50.687590 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562a7d3-a817-482c-8100-b96b9867fa0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:50.743821 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-2562a7d3-a817-482c-8100-b96b9867fa0d None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:50.747182 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2562a7d3-a817-482c-8100-b96b9867fa0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:50.809030 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c3318-ff84-487c-b9d9-922829208bb9 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:51.033795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2002826-c80c-4c5f-b97b-3167d28e992b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:51.036035 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2002826-c80c-4c5f-b97b-3167d28e992b service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:51.036035 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2002826-c80c-4c5f-b97b-3167d28e992b service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:51.036035 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2002826-c80c-4c5f-b97b-3167d28e992b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:51.045445 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 970/1969] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:51.047849 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c2002826-c80c-4c5f-b97b-3167d28e992b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:51.058507 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac0652-696f-4671-8035-cb391e976445 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:51.058981 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac0652-696f-4671-8035-cb391e976445 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:51.059470 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac0652-696f-4671-8035-cb391e976445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:51.089570 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:51.126026 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-0aac0652-696f-4671-8035-cb391e976445 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:51.130433 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0aac0652-696f-4671-8035-cb391e976445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:51.172124 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 971/1970] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:30:51.179544 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aac0652-696f-4671-8035-cb391e976445 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:51.231836 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aac0652-696f-4671-8035-cb391e976445 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:51.261817 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193c2160-bc19-428c-8a95-7d1c50b0a812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:51.263540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c2160-bc19-428c-8a95-7d1c50b0a812 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:51.269202 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c2160-bc19-428c-8a95-7d1c50b0a812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:51.270970 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c2160-bc19-428c-8a95-7d1c50b0a812 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:51.276485 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193c2160-bc19-428c-8a95-7d1c50b0a812 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:30:51.279098 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193c2160-bc19-428c-8a95-7d1c50b0a812 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:51.287855 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193c2160-bc19-428c-8a95-7d1c50b0a812 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:51.368574 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 972/1971] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:30:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:30:51.374157 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193c2160-bc19-428c-8a95-7d1c50b0a812 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:51.387407 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2002826-c80c-4c5f-b97b-3167d28e992b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:51.390590 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2002826-c80c-4c5f-b97b-3167d28e992b service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:51.429082 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193c2160-bc19-428c-8a95-7d1c50b0a812 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:51.442202 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1000/1972] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:30:50 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 837 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:30:51.486721 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2002826-c80c-4c5f-b97b-3167d28e992b service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:51.489356 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c2002826-c80c-4c5f-b97b-3167d28e992b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:51.492091 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d42a1f-27f4-4443-abe9-fadbc641b029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:51.493387 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d42a1f-27f4-4443-abe9-fadbc641b029 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:51.498282 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d42a1f-27f4-4443-abe9-fadbc641b029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:51.499948 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d42a1f-27f4-4443-abe9-fadbc641b029 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:51.502807 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d42a1f-27f4-4443-abe9-fadbc641b029 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:51.514201 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d42a1f-27f4-4443-abe9-fadbc641b029 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:51.520267 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2002826-c80c-4c5f-b97b-3167d28e992b service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:51.526968 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:51.547624 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7af133-aa32-425f-9deb-b468dd773c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:51.553083 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7af133-aa32-425f-9deb-b468dd773c86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:51.553860 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7af133-aa32-425f-9deb-b468dd773c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:51.554458 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7af133-aa32-425f-9deb-b468dd773c86 admin admin] PATH_INFO: `/v3/users/fc5d94367276402b97ad8d844ffbd5de` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:51.559414 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7af133-aa32-425f-9deb-b468dd773c86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc5d94367276402b97ad8d844ffbd5de)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:51.565765 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7af133-aa32-425f-9deb-b468dd773c86 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:51.631666 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 973/1973] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:30:51 2022] POST /identity/v3/projects => generated 385 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:30:51.639706 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe97aa4f-d139-4acb-bcd9-b8ac09b6231e None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:51.640014 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe97aa4f-d139-4acb-bcd9-b8ac09b6231e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:51.640311 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe97aa4f-d139-4acb-bcd9-b8ac09b6231e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:51.691748 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-fe97aa4f-d139-4acb-bcd9-b8ac09b6231e None None] MFA Rules not processed for user `82c323bc7a8c45e39714bb3e7f401c3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:51.694811 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fe97aa4f-d139-4acb-bcd9-b8ac09b6231e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:51.731794 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9e7af133-aa32-425f-9deb-b468dd773c86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc5d94367276402b97ad8d844ffbd5de'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:51.775968 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9e7af133-aa32-425f-9deb-b468dd773c86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc5d94367276402b97ad8d844ffbd5de'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:51.777959 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9e7af133-aa32-425f-9deb-b468dd773c86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc5d94367276402b97ad8d844ffbd5de'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:51.781005 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9e7af133-aa32-425f-9deb-b468dd773c86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc5d94367276402b97ad8d844ffbd5de'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:51.807997 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9e7af133-aa32-425f-9deb-b468dd773c86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc5d94367276402b97ad8d844ffbd5de'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:51.842096 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1001/1974] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:30:51 2022] DELETE /identity/v3/users/fc5d94367276402b97ad8d844ffbd5de => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:51.845717 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7af133-aa32-425f-9deb-b468dd773c86 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:51.856685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-9e7af133-aa32-425f-9deb-b468dd773c86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:51.961997 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 974/1975] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:51.966160 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d14eab7-7ffc-42a8-a344-606317575eef None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:51.966795 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d14eab7-7ffc-42a8-a344-606317575eef None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:51.978896 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d14eab7-7ffc-42a8-a344-606317575eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:52.045131 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3d14eab7-7ffc-42a8-a344-606317575eef None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:52.048576 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3d14eab7-7ffc-42a8-a344-606317575eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:52.083678 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7af133-aa32-425f-9deb-b468dd773c86 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:52.091851 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d06b2e6-96ca-4acd-91a9-5dd03127c7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:52.092575 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06b2e6-96ca-4acd-91a9-5dd03127c7de admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:52.092857 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06b2e6-96ca-4acd-91a9-5dd03127c7de admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:52.093300 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06b2e6-96ca-4acd-91a9-5dd03127c7de admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:52.094134 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d06b2e6-96ca-4acd-91a9-5dd03127c7de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:52.096215 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d06b2e6-96ca-4acd-91a9-5dd03127c7de admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:52.153336 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1002/1976] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:30:51 2022] POST /identity/v3/users => generated 293 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:30:52.155040 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d06b2e6-96ca-4acd-91a9-5dd03127c7de admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:52.159887 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-0d06b2e6-96ca-4acd-91a9-5dd03127c7de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:52.185591 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:52.221395 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 975/1977] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:52.223811 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d14eab7-7ffc-42a8-a344-606317575eef None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:52.239036 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d14eab7-7ffc-42a8-a344-606317575eef None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:52.247824 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc9427d-fc04-4b1a-befc-b8a2bbf14fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:52.250486 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9427d-fc04-4b1a-befc-b8a2bbf14fb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:52.250486 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9427d-fc04-4b1a-befc-b8a2bbf14fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:52.250486 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9427d-fc04-4b1a-befc-b8a2bbf14fb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:52.251748 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc9427d-fc04-4b1a-befc-b8a2bbf14fb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:52.253560 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc9427d-fc04-4b1a-befc-b8a2bbf14fb5 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:52.267646 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 976/1978] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:30:52.268298 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc9427d-fc04-4b1a-befc-b8a2bbf14fb5 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:52.270432 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6cc9427d-fc04-4b1a-befc-b8a2bbf14fb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:52.309899 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d06b2e6-96ca-4acd-91a9-5dd03127c7de admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:52.325221 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef105d4e-45f7-4e9b-b7dd-49da9b039f8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:52.327065 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef105d4e-45f7-4e9b-b7dd-49da9b039f8d service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:52.327960 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef105d4e-45f7-4e9b-b7dd-49da9b039f8d service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:52.328541 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef105d4e-45f7-4e9b-b7dd-49da9b039f8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:52.370847 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc9427d-fc04-4b1a-befc-b8a2bbf14fb5 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:52.379665 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d48f22-dfee-4001-8352-661dc3822fa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:52.381047 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d48f22-dfee-4001-8352-661dc3822fa9 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:52.382167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d48f22-dfee-4001-8352-661dc3822fa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:52.382711 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d48f22-dfee-4001-8352-661dc3822fa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:52.403039 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d48f22-dfee-4001-8352-661dc3822fa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:52.407572 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d48f22-dfee-4001-8352-661dc3822fa9 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:52.431713 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 977/1979] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:30:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:30:52.433067 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d48f22-dfee-4001-8352-661dc3822fa9 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:52.448006 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d48f22-dfee-4001-8352-661dc3822fa9 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:52.461499 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e5c65-8e8b-477c-a738-0fce94e67587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:52.462757 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef105d4e-45f7-4e9b-b7dd-49da9b039f8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:52.463424 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e5c65-8e8b-477c-a738-0fce94e67587 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:52.464188 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e5c65-8e8b-477c-a738-0fce94e67587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:52.464921 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef105d4e-45f7-4e9b-b7dd-49da9b039f8d service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:52.465199 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e5c65-8e8b-477c-a738-0fce94e67587 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:52.466615 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373e5c65-8e8b-477c-a738-0fce94e67587 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:52.469541 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373e5c65-8e8b-477c-a738-0fce94e67587 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:52.473113 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:52.484215 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1003/1980] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:30:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:30:52.511168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 978/1981] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:30:52.518417 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef105d4e-45f7-4e9b-b7dd-49da9b039f8d service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:52.545325 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef105d4e-45f7-4e9b-b7dd-49da9b039f8d service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:52.569002 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2825af04-aa95-4efb-bc76-e67111e83b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:52.570658 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2825af04-aa95-4efb-bc76-e67111e83b49 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:52.571178 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2825af04-aa95-4efb-bc76-e67111e83b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:52.571622 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2825af04-aa95-4efb-bc76-e67111e83b49 admin admin] PATH_INFO: `/v3/projects/9fd9c0e3948f41ce81b58167c73cc7c3/users/b459e85999664ee588b91400f93825c3/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:52.619859 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2825af04-aa95-4efb-bc76-e67111e83b49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fd9c0e3948f41ce81b58167c73cc7c3, user_id=b459e85999664ee588b91400f93825c3, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:52.627035 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2825af04-aa95-4efb-bc76-e67111e83b49 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:52.659038 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1004/1982] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:30:52 2022] PUT /identity/v3/projects/9fd9c0e3948f41ce81b58167c73cc7c3/users/b459e85999664ee588b91400f93825c3/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:30:52.696583 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee5ad6-4d1c-417d-baa3-69377589325c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:52.698143 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee5ad6-4d1c-417d-baa3-69377589325c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:52.699077 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee5ad6-4d1c-417d-baa3-69377589325c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:52.773546 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d7ee5ad6-4d1c-417d-baa3-69377589325c None None] MFA Rules not processed for user `b459e85999664ee588b91400f93825c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:52.775274 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d7ee5ad6-4d1c-417d-baa3-69377589325c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:52.987994 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 979/1983] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:52.996801 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2825af04-aa95-4efb-bc76-e67111e83b49 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:53.143610 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2825af04-aa95-4efb-bc76-e67111e83b49 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:53.154567 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546744da-a3e7-4f0b-a4d4-0d56ac9ff9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:53.155752 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546744da-a3e7-4f0b-a4d4-0d56ac9ff9d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:53.158222 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546744da-a3e7-4f0b-a4d4-0d56ac9ff9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:53.158766 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546744da-a3e7-4f0b-a4d4-0d56ac9ff9d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:53.161929 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546744da-a3e7-4f0b-a4d4-0d56ac9ff9d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:53.165925 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546744da-a3e7-4f0b-a4d4-0d56ac9ff9d0 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:53.228531 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1005/1984] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:30:52 2022] POST /identity/v3/projects => generated 387 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:30:53.238677 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ee5ad6-4d1c-417d-baa3-69377589325c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:53.264915 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ee5ad6-4d1c-417d-baa3-69377589325c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:53.456073 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d207541-2f36-4884-bb62-e8fc1e42855e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:53.457302 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d207541-2f36-4884-bb62-e8fc1e42855e admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:53.457846 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d207541-2f36-4884-bb62-e8fc1e42855e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:53.458351 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d207541-2f36-4884-bb62-e8fc1e42855e admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:53.460938 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d207541-2f36-4884-bb62-e8fc1e42855e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:53.464935 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d207541-2f36-4884-bb62-e8fc1e42855e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:53.544694 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 980/1985] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:30:53 2022] POST /identity/v3/users => generated 300 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:30:53.545154 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546744da-a3e7-4f0b-a4d4-0d56ac9ff9d0 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:53.571727 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546744da-a3e7-4f0b-a4d4-0d56ac9ff9d0 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:53.593724 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b6ad48-792a-4b74-b86d-392aad712997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:53.593724 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b6ad48-792a-4b74-b86d-392aad712997 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:53.602184 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b6ad48-792a-4b74-b86d-392aad712997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:53.602184 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b6ad48-792a-4b74-b86d-392aad712997 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:53.602184 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b6ad48-792a-4b74-b86d-392aad712997 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:53.602184 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b6ad48-792a-4b74-b86d-392aad712997 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:53.644335 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1006/1986] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:53 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:30:53.651384 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d207541-2f36-4884-bb62-e8fc1e42855e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:53.677897 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d207541-2f36-4884-bb62-e8fc1e42855e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:53.685901 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb5bc4-af30-4f0d-907f-6a230589d4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:53.687510 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb5bc4-af30-4f0d-907f-6a230589d4f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:53.687896 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb5bc4-af30-4f0d-907f-6a230589d4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:53.688696 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb5bc4-af30-4f0d-907f-6a230589d4f9 admin admin] PATH_INFO: `/v3/projects/76eb64a1062a49a79465ad793b7cd3ec/users/18e6eecfae1146758bb774f940233c22/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:53.719209 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddb5bc4-af30-4f0d-907f-6a230589d4f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76eb64a1062a49a79465ad793b7cd3ec, user_id=18e6eecfae1146758bb774f940233c22, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:53.725952 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddb5bc4-af30-4f0d-907f-6a230589d4f9 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:53.771715 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 981/1987] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:30:53 2022] PUT /identity/v3/projects/76eb64a1062a49a79465ad793b7cd3ec/users/18e6eecfae1146758bb774f940233c22/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:53.777794 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b6ad48-792a-4b74-b86d-392aad712997 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:53.788774 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb5bc4-af30-4f0d-907f-6a230589d4f9 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:53.795220 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b6ad48-792a-4b74-b86d-392aad712997 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:53.795795 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-cddb5bc4-af30-4f0d-907f-6a230589d4f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:53.832126 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:53.968396 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a9e21b-fa7f-4a98-a5c1-3d32dc02ab74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:53.970221 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a9e21b-fa7f-4a98-a5c1-3d32dc02ab74 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:53.971147 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a9e21b-fa7f-4a98-a5c1-3d32dc02ab74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:53.971822 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a9e21b-fa7f-4a98-a5c1-3d32dc02ab74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:53.974153 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a9e21b-fa7f-4a98-a5c1-3d32dc02ab74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:53.976999 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a9e21b-fa7f-4a98-a5c1-3d32dc02ab74 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:54.000934 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1007/1988] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:53 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:30:54.007050 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a9e21b-fa7f-4a98-a5c1-3d32dc02ab74 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:54.010286 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:54.043084 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb5bc4-af30-4f0d-907f-6a230589d4f9 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:54.043348 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a9e21b-fa7f-4a98-a5c1-3d32dc02ab74 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:54.051475 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a340622-b6a5-4c9b-9bf3-935a983cd4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:54.053060 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a340622-b6a5-4c9b-9bf3-935a983cd4be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:54.053619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a340622-b6a5-4c9b-9bf3-935a983cd4be admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:54.054153 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a340622-b6a5-4c9b-9bf3-935a983cd4be admin admin] PATH_INFO: `/v3/projects/6dfc97ee2ef947bd868e00767d3ece77` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:54.055179 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef7b4d-0326-4ccb-9825-4cc27f916801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:54.056242 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a340622-b6a5-4c9b-9bf3-935a983cd4be admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dfc97ee2ef947bd868e00767d3ece77)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:54.056811 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef7b4d-0326-4ccb-9825-4cc27f916801 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:54.057504 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef7b4d-0326-4ccb-9825-4cc27f916801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:54.058073 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef7b4d-0326-4ccb-9825-4cc27f916801 admin admin] PATH_INFO: `/v3/projects/76eb64a1062a49a79465ad793b7cd3ec/users/18e6eecfae1146758bb774f940233c22/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:54.061747 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a340622-b6a5-4c9b-9bf3-935a983cd4be admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:54.064192 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bef7b4d-0326-4ccb-9825-4cc27f916801 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76eb64a1062a49a79465ad793b7cd3ec, user_id=18e6eecfae1146758bb774f940233c22, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:54.070142 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bef7b4d-0326-4ccb-9825-4cc27f916801 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:54.113686 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1008/1989] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:30:54 2022] PUT /identity/v3/projects/76eb64a1062a49a79465ad793b7cd3ec/users/18e6eecfae1146758bb774f940233c22/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:54.118992 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece1c099-168f-40e4-b3af-bc3b73187d8b None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:54.119915 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece1c099-168f-40e4-b3af-bc3b73187d8b None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:54.120572 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece1c099-168f-40e4-b3af-bc3b73187d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:54.241827 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-ece1c099-168f-40e4-b3af-bc3b73187d8b None None] MFA Rules not processed for user `18e6eecfae1146758bb774f940233c22`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:54.242109 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7a340622-b6a5-4c9b-9bf3-935a983cd4be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dfc97ee2ef947bd868e00767d3ece77', 'request_id': 'req-7a340622-b6a5-4c9b-9bf3-935a983cd4be'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:54.245566 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ece1c099-168f-40e4-b3af-bc3b73187d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:54.282989 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7a340622-b6a5-4c9b-9bf3-935a983cd4be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dfc97ee2ef947bd868e00767d3ece77', 'request_id': 'req-7a340622-b6a5-4c9b-9bf3-935a983cd4be'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:54.336699 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7a340622-b6a5-4c9b-9bf3-935a983cd4be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dfc97ee2ef947bd868e00767d3ece77', 'request_id': 'req-7a340622-b6a5-4c9b-9bf3-935a983cd4be'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:54.371306 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7a340622-b6a5-4c9b-9bf3-935a983cd4be admin admin] The token cache is being invalidate because project 6dfc97ee2ef947bd868e00767d3ece77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:30:54.371852 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7a340622-b6a5-4c9b-9bf3-935a983cd4be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:54.373170 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7a340622-b6a5-4c9b-9bf3-935a983cd4be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:54.377705 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 982/1990] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:30:53 2022] DELETE /identity/v3/projects/6dfc97ee2ef947bd868e00767d3ece77 => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:54.385021 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf6ac9c-3f0f-4c42-843b-f3c9c7a289a4 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:54.385580 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf6ac9c-3f0f-4c42-843b-f3c9c7a289a4 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:54.386074 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf6ac9c-3f0f-4c42-843b-f3c9c7a289a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:54.461541 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-4bf6ac9c-3f0f-4c42-843b-f3c9c7a289a4 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:54.468140 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4bf6ac9c-3f0f-4c42-843b-f3c9c7a289a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:54.498208 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1009/1991] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:54 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:30:54.528089 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece1c099-168f-40e4-b3af-bc3b73187d8b None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:54.531401 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ece1c099-168f-40e4-b3af-bc3b73187d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:54.724960 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 983/1992] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:30:54.738865 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bffc2e-8eac-4561-92e3-514abe1b8446 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:54.739508 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bffc2e-8eac-4561-92e3-514abe1b8446 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:54.740085 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bffc2e-8eac-4561-92e3-514abe1b8446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:54.795845 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-61bffc2e-8eac-4561-92e3-514abe1b8446 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:54.799832 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-61bffc2e-8eac-4561-92e3-514abe1b8446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:54.815504 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece1c099-168f-40e4-b3af-bc3b73187d8b None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:54.828250 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 984/1993] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:54.830661 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e155f328-0660-4bfa-813b-cdffc7b1dbce None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:54.831127 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e155f328-0660-4bfa-813b-cdffc7b1dbce None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:54.831576 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e155f328-0660-4bfa-813b-cdffc7b1dbce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:54.858991 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95299b58-32b0-4fab-97d0-e7d0febaaae8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:54.861211 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95299b58-32b0-4fab-97d0-e7d0febaaae8 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:54.862129 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95299b58-32b0-4fab-97d0-e7d0febaaae8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:54.863265 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95299b58-32b0-4fab-97d0-e7d0febaaae8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:54.881914 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95299b58-32b0-4fab-97d0-e7d0febaaae8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:54.886116 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95299b58-32b0-4fab-97d0-e7d0febaaae8 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:54.908167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-e155f328-0660-4bfa-813b-cdffc7b1dbce None None] MFA Rules not processed for user `82c323bc7a8c45e39714bb3e7f401c3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:54.915990 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e155f328-0660-4bfa-813b-cdffc7b1dbce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:54.928819 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1010/1994] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:30:54 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:30:54.930486 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95299b58-32b0-4fab-97d0-e7d0febaaae8 service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:54.962119 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95299b58-32b0-4fab-97d0-e7d0febaaae8 service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:54.970506 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3111ae-2420-4712-97c0-6f5bfdcd5ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:54.971426 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3111ae-2420-4712-97c0-6f5bfdcd5ccf admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:54.971818 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3111ae-2420-4712-97c0-6f5bfdcd5ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:54.972225 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3111ae-2420-4712-97c0-6f5bfdcd5ccf admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:54.973402 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3111ae-2420-4712-97c0-6f5bfdcd5ccf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:30:54.973973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3111ae-2420-4712-97c0-6f5bfdcd5ccf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:54.976338 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3111ae-2420-4712-97c0-6f5bfdcd5ccf admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:55.040152 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1011/1995] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:30:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:30:55.051636 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3111ae-2420-4712-97c0-6f5bfdcd5ccf admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:55.143636 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3111ae-2420-4712-97c0-6f5bfdcd5ccf admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:55.174161 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d629e39d-84dc-4752-8766-62a3dbd46965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:55.174711 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d629e39d-84dc-4752-8766-62a3dbd46965 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:55.175209 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d629e39d-84dc-4752-8766-62a3dbd46965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:55.175740 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d629e39d-84dc-4752-8766-62a3dbd46965 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:55.183997 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d629e39d-84dc-4752-8766-62a3dbd46965 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:55.194480 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d629e39d-84dc-4752-8766-62a3dbd46965 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:55.201070 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 985/1996] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:55.226678 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:55.228705 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e155f328-0660-4bfa-813b-cdffc7b1dbce None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:55.235366 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e155f328-0660-4bfa-813b-cdffc7b1dbce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:55.296324 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e155f328-0660-4bfa-813b-cdffc7b1dbce None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:55.300735 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1012/1997] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:30:55 2022] POST /identity/v3/projects => generated 373 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:30:55.302200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d629e39d-84dc-4752-8766-62a3dbd46965 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:55.368690 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:55.548322 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b168d-1ae5-41e6-b78f-aa3650023888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:55.552695 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b168d-1ae5-41e6-b78f-aa3650023888 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:55.553459 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b168d-1ae5-41e6-b78f-aa3650023888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:55.554114 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b168d-1ae5-41e6-b78f-aa3650023888 admin admin] PATH_INFO: `/v3/users/b459e85999664ee588b91400f93825c3` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:55.560854 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5b168d-1ae5-41e6-b78f-aa3650023888 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b459e85999664ee588b91400f93825c3)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:55.565009 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5b168d-1ae5-41e6-b78f-aa3650023888 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:55.599510 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d629e39d-84dc-4752-8766-62a3dbd46965 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:55.611164 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c332a1-bc07-492c-b83d-81cfd5ed0975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:55.612322 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c332a1-bc07-492c-b83d-81cfd5ed0975 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:55.612824 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c332a1-bc07-492c-b83d-81cfd5ed0975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:55.613341 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c332a1-bc07-492c-b83d-81cfd5ed0975 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:55.614779 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c332a1-bc07-492c-b83d-81cfd5ed0975 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:55.620225 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c332a1-bc07-492c-b83d-81cfd5ed0975 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:55.668371 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1013/1998] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:30:55 2022] POST /identity/v3/users => generated 287 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:30:55.670450 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c332a1-bc07-492c-b83d-81cfd5ed0975 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:55.684155 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ad5b168d-1ae5-41e6-b78f-aa3650023888 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b459e85999664ee588b91400f93825c3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:55.704650 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ad5b168d-1ae5-41e6-b78f-aa3650023888 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b459e85999664ee588b91400f93825c3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:55.706109 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ad5b168d-1ae5-41e6-b78f-aa3650023888 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b459e85999664ee588b91400f93825c3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:55.706770 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c332a1-bc07-492c-b83d-81cfd5ed0975 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:55.708188 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ad5b168d-1ae5-41e6-b78f-aa3650023888 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b459e85999664ee588b91400f93825c3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:55.722857 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98a729b-2f8d-4d1b-8ea3-872274c95994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:55.725123 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a729b-2f8d-4d1b-8ea3-872274c95994 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:55.725453 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a729b-2f8d-4d1b-8ea3-872274c95994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:55.726848 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a729b-2f8d-4d1b-8ea3-872274c95994 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:55.731601 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98a729b-2f8d-4d1b-8ea3-872274c95994 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:55.738969 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98a729b-2f8d-4d1b-8ea3-872274c95994 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:55.739852 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ad5b168d-1ae5-41e6-b78f-aa3650023888 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b459e85999664ee588b91400f93825c3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:55.771169 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1014/1999] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:55 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:30:55.775926 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98a729b-2f8d-4d1b-8ea3-872274c95994 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:55.790874 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 986/2000] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:30:55 2022] DELETE /identity/v3/users/b459e85999664ee588b91400f93825c3 => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:55.791268 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b98a729b-2f8d-4d1b-8ea3-872274c95994 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:55.833838 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f0d6b-9b36-4a80-9a35-df5b326037ae None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:55.834327 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98a729b-2f8d-4d1b-8ea3-872274c95994 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:55.835012 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f0d6b-9b36-4a80-9a35-df5b326037ae None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:55.835761 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f0d6b-9b36-4a80-9a35-df5b326037ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:55.895854 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-6b3f0d6b-9b36-4a80-9a35-df5b326037ae None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:55.898341 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6b3f0d6b-9b36-4a80-9a35-df5b326037ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:56.099961 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 987/2001] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:56.148595 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d48a97-80b5-40b6-990e-1155335887a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:56.149420 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d48a97-80b5-40b6-990e-1155335887a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:56.149760 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d48a97-80b5-40b6-990e-1155335887a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:56.149907 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d48a97-80b5-40b6-990e-1155335887a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:56.150696 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d48a97-80b5-40b6-990e-1155335887a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:56.152547 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d48a97-80b5-40b6-990e-1155335887a2 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:56.190934 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1015/2002] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:55 2022] GET /identity/v3/roles => generated 1420 bytes in 410 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:30:56.194891 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f0d6b-9b36-4a80-9a35-df5b326037ae None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:56.215377 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d48a97-80b5-40b6-990e-1155335887a2 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:56.223113 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b6d48a97-80b5-40b6-990e-1155335887a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:56.225154 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f0d6b-9b36-4a80-9a35-df5b326037ae None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:56.247247 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe239db-98f0-419d-9124-16bdaecd862c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:56.248611 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe239db-98f0-419d-9124-16bdaecd862c admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:56.249099 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe239db-98f0-419d-9124-16bdaecd862c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:56.250105 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe239db-98f0-419d-9124-16bdaecd862c admin admin] PATH_INFO: `/v3/projects/1c586c98abd2438e96a5f3c56c4b476d/users/bc964b017d2f44bbbcaf06684b213e23/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:56.306970 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe239db-98f0-419d-9124-16bdaecd862c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c586c98abd2438e96a5f3c56c4b476d, user_id=bc964b017d2f44bbbcaf06684b213e23, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:56.314836 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe239db-98f0-419d-9124-16bdaecd862c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:56.370578 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 988/2003] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:30:56 2022] PUT /identity/v3/projects/1c586c98abd2438e96a5f3c56c4b476d/users/bc964b017d2f44bbbcaf06684b213e23/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:56.380460 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354a7ec-2cab-4677-a178-d2e43204ba88 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:56.382828 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354a7ec-2cab-4677-a178-d2e43204ba88 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:56.383946 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354a7ec-2cab-4677-a178-d2e43204ba88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:56.504997 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-5354a7ec-2cab-4677-a178-d2e43204ba88 None None] MFA Rules not processed for user `bc964b017d2f44bbbcaf06684b213e23`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:56.508901 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5354a7ec-2cab-4677-a178-d2e43204ba88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:56.511606 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d48a97-80b5-40b6-990e-1155335887a2 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:56.519742 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:56.594245 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:56.799294 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b852a8ea-e09b-4ef7-b35f-5c655fd85a19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:56.801515 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852a8ea-e09b-4ef7-b35f-5c655fd85a19 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:56.804035 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852a8ea-e09b-4ef7-b35f-5c655fd85a19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:56.805284 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852a8ea-e09b-4ef7-b35f-5c655fd85a19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:56.816485 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 989/2004] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 440 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:30:56.821430 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5354a7ec-2cab-4677-a178-d2e43204ba88 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:56.829254 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5354a7ec-2cab-4677-a178-d2e43204ba88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:57.033812 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b852a8ea-e09b-4ef7-b35f-5c655fd85a19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:57.037049 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b852a8ea-e09b-4ef7-b35f-5c655fd85a19 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:57.052697 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5354a7ec-2cab-4677-a178-d2e43204ba88 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:57.062365 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0e5bb2-a902-4b93-84d5-2cec9df59a9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:57.063076 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e5bb2-a902-4b93-84d5-2cec9df59a9a service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:57.063370 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e5bb2-a902-4b93-84d5-2cec9df59a9a service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:57.063654 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e5bb2-a902-4b93-84d5-2cec9df59a9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:57.069090 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2f0e5bb2-a902-4b93-84d5-2cec9df59a9a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:57.078400 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1016/2005] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:30:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 865 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:30:57.080588 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b852a8ea-e09b-4ef7-b35f-5c655fd85a19 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:57.101788 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b852a8ea-e09b-4ef7-b35f-5c655fd85a19 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:57.132051 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c7de8-a2d7-4e9f-9387-4fd637d9e8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:57.133109 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c7de8-a2d7-4e9f-9387-4fd637d9e8fc admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:57.133785 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c7de8-a2d7-4e9f-9387-4fd637d9e8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:57.134403 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c7de8-a2d7-4e9f-9387-4fd637d9e8fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:57.135741 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260c7de8-a2d7-4e9f-9387-4fd637d9e8fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:57.139985 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260c7de8-a2d7-4e9f-9387-4fd637d9e8fc admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:57.214032 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1017/2006] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:30:57 2022] POST /identity/v3/projects => generated 373 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:30:57.215764 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c7de8-a2d7-4e9f-9387-4fd637d9e8fc admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:57.351479 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0e5bb2-a902-4b93-84d5-2cec9df59a9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:57.359964 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0e5bb2-a902-4b93-84d5-2cec9df59a9a service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:57.383460 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 990/2007] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:30:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:30:57.462338 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c7de8-a2d7-4e9f-9387-4fd637d9e8fc admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:57.491199 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c18cb54-76f1-4dee-be50-e517b36c316e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:57.493362 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c18cb54-76f1-4dee-be50-e517b36c316e admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:57.498199 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c18cb54-76f1-4dee-be50-e517b36c316e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:57.498960 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c18cb54-76f1-4dee-be50-e517b36c316e admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:57.500128 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c18cb54-76f1-4dee-be50-e517b36c316e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:57.505965 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c18cb54-76f1-4dee-be50-e517b36c316e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:57.617358 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1018/2008] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:30:57 2022] POST /identity/v3/users => generated 293 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:30:57.624736 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0e5bb2-a902-4b93-84d5-2cec9df59a9a service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:57.652646 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0e5bb2-a902-4b93-84d5-2cec9df59a9a service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:57.659632 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53faa52c-d61d-4087-8b92-6e20aff4f816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:57.660357 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53faa52c-d61d-4087-8b92-6e20aff4f816 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:57.660639 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53faa52c-d61d-4087-8b92-6e20aff4f816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:57.660917 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53faa52c-d61d-4087-8b92-6e20aff4f816 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:57.661702 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53faa52c-d61d-4087-8b92-6e20aff4f816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:57.664327 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53faa52c-d61d-4087-8b92-6e20aff4f816 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:57.708776 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c18cb54-76f1-4dee-be50-e517b36c316e admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:57.719243 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 991/2009] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:57 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:30:57.739972 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c18cb54-76f1-4dee-be50-e517b36c316e admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:57.754591 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c9de58-1034-47b0-afb8-78d10e075fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:57.755719 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c9de58-1034-47b0-afb8-78d10e075fc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:57.756049 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c9de58-1034-47b0-afb8-78d10e075fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:57.756331 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c9de58-1034-47b0-afb8-78d10e075fc2 admin admin] PATH_INFO: `/v3/projects/65d6599fa1f546239055f1d44f2ae644/users/b159bd7b405246beacec59da6b5c44a6/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:57.822781 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c9de58-1034-47b0-afb8-78d10e075fc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65d6599fa1f546239055f1d44f2ae644, user_id=b159bd7b405246beacec59da6b5c44a6, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:57.847137 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c9de58-1034-47b0-afb8-78d10e075fc2 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:57.907137 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1019/2010] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:30:57 2022] PUT /identity/v3/projects/65d6599fa1f546239055f1d44f2ae644/users/b159bd7b405246beacec59da6b5c44a6/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:30:57.910205 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53faa52c-d61d-4087-8b92-6e20aff4f816 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:57.943802 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53faa52c-d61d-4087-8b92-6e20aff4f816 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:57.994164 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c9de58-1034-47b0-afb8-78d10e075fc2 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:57.997054 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-12c9de58-1034-47b0-afb8-78d10e075fc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:58.046744 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:58.124869 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:58.154766 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83590993-2607-4736-af0d-0c367ba53467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:58.157069 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83590993-2607-4736-af0d-0c367ba53467 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:58.158309 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83590993-2607-4736-af0d-0c367ba53467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:58.158715 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83590993-2607-4736-af0d-0c367ba53467 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:58.161443 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83590993-2607-4736-af0d-0c367ba53467 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:58.165897 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83590993-2607-4736-af0d-0c367ba53467 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:58.195375 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c9de58-1034-47b0-afb8-78d10e075fc2 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:58.199454 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 992/2011] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:57 2022] GET /identity/v3/roles => generated 1420 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:30:58.200840 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83590993-2607-4736-af0d-0c367ba53467 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:58.205452 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba66924-eb0d-44b6-a260-ecd425e8d97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:58.207687 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba66924-eb0d-44b6-a260-ecd425e8d97f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:58.208061 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba66924-eb0d-44b6-a260-ecd425e8d97f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:58.208442 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba66924-eb0d-44b6-a260-ecd425e8d97f admin admin] PATH_INFO: `/v3/projects/9fd9c0e3948f41ce81b58167c73cc7c3` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:58.211912 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba66924-eb0d-44b6-a260-ecd425e8d97f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fd9c0e3948f41ce81b58167c73cc7c3)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:58.214718 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba66924-eb0d-44b6-a260-ecd425e8d97f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:58.233269 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83590993-2607-4736-af0d-0c367ba53467 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:58.243640 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509e872b-8133-4be9-8a57-f4f6b445f6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:58.245346 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e872b-8133-4be9-8a57-f4f6b445f6e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:58.245879 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e872b-8133-4be9-8a57-f4f6b445f6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:58.246360 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e872b-8133-4be9-8a57-f4f6b445f6e1 admin admin] PATH_INFO: `/v3/projects/65d6599fa1f546239055f1d44f2ae644/users/b159bd7b405246beacec59da6b5c44a6/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:58.251574 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509e872b-8133-4be9-8a57-f4f6b445f6e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65d6599fa1f546239055f1d44f2ae644, user_id=b159bd7b405246beacec59da6b5c44a6, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:58.257125 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509e872b-8133-4be9-8a57-f4f6b445f6e1 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:58.309773 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 993/2012] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:30:58 2022] PUT /identity/v3/projects/65d6599fa1f546239055f1d44f2ae644/users/b159bd7b405246beacec59da6b5c44a6/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:58.340504 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000226bc-a984-4045-b02f-3f9a38728936 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:58.340808 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000226bc-a984-4045-b02f-3f9a38728936 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:58.352531 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000226bc-a984-4045-b02f-3f9a38728936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:58.399036 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5ba66924-eb0d-44b6-a260-ecd425e8d97f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fd9c0e3948f41ce81b58167c73cc7c3', 'request_id': 'req-5ba66924-eb0d-44b6-a260-ecd425e8d97f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:58.458111 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5ba66924-eb0d-44b6-a260-ecd425e8d97f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fd9c0e3948f41ce81b58167c73cc7c3', 'request_id': 'req-5ba66924-eb0d-44b6-a260-ecd425e8d97f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:58.466915 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-000226bc-a984-4045-b02f-3f9a38728936 None None] MFA Rules not processed for user `b159bd7b405246beacec59da6b5c44a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:58.469412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-000226bc-a984-4045-b02f-3f9a38728936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:58.484971 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5ba66924-eb0d-44b6-a260-ecd425e8d97f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fd9c0e3948f41ce81b58167c73cc7c3', 'request_id': 'req-5ba66924-eb0d-44b6-a260-ecd425e8d97f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:58.510854 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5ba66924-eb0d-44b6-a260-ecd425e8d97f admin admin] The token cache is being invalidate because project 9fd9c0e3948f41ce81b58167c73cc7c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:30:58.511242 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5ba66924-eb0d-44b6-a260-ecd425e8d97f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:58.512270 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5ba66924-eb0d-44b6-a260-ecd425e8d97f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:58.524402 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1020/2013] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:30:57 2022] DELETE /identity/v3/projects/9fd9c0e3948f41ce81b58167c73cc7c3 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:30:58.524402 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba66924-eb0d-44b6-a260-ecd425e8d97f admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:58.525445 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5ba66924-eb0d-44b6-a260-ecd425e8d97f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:58.726584 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba66924-eb0d-44b6-a260-ecd425e8d97f admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:58.729925 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 994/2014] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:30:58 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:30:58.748575 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000226bc-a984-4045-b02f-3f9a38728936 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:58.751015 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-000226bc-a984-4045-b02f-3f9a38728936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:58.753505 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4931cc4f-44c3-4276-81f1-b1031b9f3211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:58.755511 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4931cc4f-44c3-4276-81f1-b1031b9f3211 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:58.756142 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4931cc4f-44c3-4276-81f1-b1031b9f3211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:58.756733 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4931cc4f-44c3-4276-81f1-b1031b9f3211 admin admin] PATH_INFO: `/v3/users/18e6eecfae1146758bb774f940233c22` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:58.759261 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4931cc4f-44c3-4276-81f1-b1031b9f3211 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18e6eecfae1146758bb774f940233c22)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:58.764005 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4931cc4f-44c3-4276-81f1-b1031b9f3211 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:58.943504 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000226bc-a984-4045-b02f-3f9a38728936 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:58.945760 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:58.954816 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-4931cc4f-44c3-4276-81f1-b1031b9f3211 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18e6eecfae1146758bb774f940233c22'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:58.965707 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ebf077-0689-463c-9293-7f72d7fe2b4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:58.966840 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebf077-0689-463c-9293-7f72d7fe2b4b service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:58.967328 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebf077-0689-463c-9293-7f72d7fe2b4b service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:58.967821 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebf077-0689-463c-9293-7f72d7fe2b4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:58.978539 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f7ebf077-0689-463c-9293-7f72d7fe2b4b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:58.999233 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-4931cc4f-44c3-4276-81f1-b1031b9f3211 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18e6eecfae1146758bb774f940233c22'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:59.000679 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-4931cc4f-44c3-4276-81f1-b1031b9f3211 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18e6eecfae1146758bb774f940233c22'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:59.001870 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-4931cc4f-44c3-4276-81f1-b1031b9f3211 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18e6eecfae1146758bb774f940233c22'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:59.015686 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-4931cc4f-44c3-4276-81f1-b1031b9f3211 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18e6eecfae1146758bb774f940233c22'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:59.034483 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1021/2015] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:30:58 2022] DELETE /identity/v3/users/18e6eecfae1146758bb774f940233c22 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:59.142361 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ebf077-0689-463c-9293-7f72d7fe2b4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:59.145545 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ebf077-0689-463c-9293-7f72d7fe2b4b service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:59.194156 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 995/2016] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:30:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:31:00.137587 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4931cc4f-44c3-4276-81f1-b1031b9f3211 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:00.139868 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4931cc4f-44c3-4276-81f1-b1031b9f3211 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:00.321953 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4931cc4f-44c3-4276-81f1-b1031b9f3211 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:00.331642 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e71e0-5e78-4fcc-be68-7caf3abc9e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:00.332781 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e71e0-5e78-4fcc-be68-7caf3abc9e16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:00.333293 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e71e0-5e78-4fcc-be68-7caf3abc9e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:00.333799 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e71e0-5e78-4fcc-be68-7caf3abc9e16 admin admin] PATH_INFO: `/v3/projects/76eb64a1062a49a79465ad793b7cd3ec` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:00.336027 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892e71e0-5e78-4fcc-be68-7caf3abc9e16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76eb64a1062a49a79465ad793b7cd3ec)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:00.340011 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892e71e0-5e78-4fcc-be68-7caf3abc9e16 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:00.459782 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-892e71e0-5e78-4fcc-be68-7caf3abc9e16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76eb64a1062a49a79465ad793b7cd3ec', 'request_id': 'req-892e71e0-5e78-4fcc-be68-7caf3abc9e16'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:00.471554 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-892e71e0-5e78-4fcc-be68-7caf3abc9e16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76eb64a1062a49a79465ad793b7cd3ec', 'request_id': 'req-892e71e0-5e78-4fcc-be68-7caf3abc9e16'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:00.506036 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-892e71e0-5e78-4fcc-be68-7caf3abc9e16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76eb64a1062a49a79465ad793b7cd3ec', 'request_id': 'req-892e71e0-5e78-4fcc-be68-7caf3abc9e16'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:00.528973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-892e71e0-5e78-4fcc-be68-7caf3abc9e16 admin admin] The token cache is being invalidate because project 76eb64a1062a49a79465ad793b7cd3ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:31:00.529367 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-892e71e0-5e78-4fcc-be68-7caf3abc9e16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:00.530461 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-892e71e0-5e78-4fcc-be68-7caf3abc9e16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:00.534717 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1022/2017] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:31:00 2022] DELETE /identity/v3/projects/76eb64a1062a49a79465ad793b7cd3ec => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:31:00.543770 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f6d5b-7fca-4487-b7ec-572eec3082fb None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:00.545580 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f6d5b-7fca-4487-b7ec-572eec3082fb None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:00.546084 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f6d5b-7fca-4487-b7ec-572eec3082fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:00.604273 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-bb2f6d5b-7fca-4487-b7ec-572eec3082fb None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:00.608587 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-bb2f6d5b-7fca-4487-b7ec-572eec3082fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:00.632752 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:00.813412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 996/2018] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:31:00.824552 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b78b7a-b8e7-4e7c-8d2b-385290b9bb17 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:00.825018 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b78b7a-b8e7-4e7c-8d2b-385290b9bb17 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:00.825505 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b78b7a-b8e7-4e7c-8d2b-385290b9bb17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:00.876481 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-48b78b7a-b8e7-4e7c-8d2b-385290b9bb17 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:00.880178 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-48b78b7a-b8e7-4e7c-8d2b-385290b9bb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:00.905038 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1023/2019] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:31:00.921528 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2f6d5b-7fca-4487-b7ec-572eec3082fb None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:00.943913 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2f6d5b-7fca-4487-b7ec-572eec3082fb None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:00.953684 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6982ca7-c0cd-4ab0-9223-abbc86b17615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:00.955216 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6982ca7-c0cd-4ab0-9223-abbc86b17615 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:00.955477 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6982ca7-c0cd-4ab0-9223-abbc86b17615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:00.955973 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6982ca7-c0cd-4ab0-9223-abbc86b17615 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:00.959266 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6982ca7-c0cd-4ab0-9223-abbc86b17615 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:31:00.959266 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6982ca7-c0cd-4ab0-9223-abbc86b17615 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:00.963406 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6982ca7-c0cd-4ab0-9223-abbc86b17615 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:00.998709 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 997/2020] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:31:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:31:01.002816 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b78b7a-b8e7-4e7c-8d2b-385290b9bb17 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:01.024564 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b78b7a-b8e7-4e7c-8d2b-385290b9bb17 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:01.033564 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750143f7-1ac5-4a33-bb53-ae82ac181689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:01.036256 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750143f7-1ac5-4a33-bb53-ae82ac181689 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:01.037122 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750143f7-1ac5-4a33-bb53-ae82ac181689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:01.037987 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750143f7-1ac5-4a33-bb53-ae82ac181689 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:01.039533 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750143f7-1ac5-4a33-bb53-ae82ac181689 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:01.044314 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750143f7-1ac5-4a33-bb53-ae82ac181689 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:01.121429 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1024/2021] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:31:00 2022] POST /identity/v3/projects => generated 389 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:31:01.123157 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6982ca7-c0cd-4ab0-9223-abbc86b17615 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:01.148905 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6982ca7-c0cd-4ab0-9223-abbc86b17615 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:01.295596 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a930c2-6db6-4323-82fa-eda77d7187f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:01.297581 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a930c2-6db6-4323-82fa-eda77d7187f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:01.302594 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a930c2-6db6-4323-82fa-eda77d7187f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:01.302594 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a930c2-6db6-4323-82fa-eda77d7187f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:01.305498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a930c2-6db6-4323-82fa-eda77d7187f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:01.313370 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a930c2-6db6-4323-82fa-eda77d7187f6 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:01.362836 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 998/2022] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:31:01 2022] POST /identity/v3/users => generated 295 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:31:01.369192 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750143f7-1ac5-4a33-bb53-ae82ac181689 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:01.399435 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750143f7-1ac5-4a33-bb53-ae82ac181689 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:01.413592 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0247dd73-4818-4107-8e33-0a4051cfb392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:01.414822 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247dd73-4818-4107-8e33-0a4051cfb392 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:01.415337 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247dd73-4818-4107-8e33-0a4051cfb392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:01.415863 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247dd73-4818-4107-8e33-0a4051cfb392 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:01.418139 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0247dd73-4818-4107-8e33-0a4051cfb392 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:01.429283 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0247dd73-4818-4107-8e33-0a4051cfb392 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:01.491012 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a930c2-6db6-4323-82fa-eda77d7187f6 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:01.503825 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1025/2023] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:31:01 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:31:01.524886 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a930c2-6db6-4323-82fa-eda77d7187f6 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:01.542944 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e20023-94c2-422b-9056-4696c16eab0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:01.544270 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e20023-94c2-422b-9056-4696c16eab0c admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:01.545887 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e20023-94c2-422b-9056-4696c16eab0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:01.545887 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e20023-94c2-422b-9056-4696c16eab0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:01.547849 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e20023-94c2-422b-9056-4696c16eab0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:01.551496 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e20023-94c2-422b-9056-4696c16eab0c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:01.573893 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 999/2024] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:31:01 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:31:01.580950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0247dd73-4818-4107-8e33-0a4051cfb392 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:01.592886 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e20023-94c2-422b-9056-4696c16eab0c admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:01.595821 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-37e20023-94c2-422b-9056-4696c16eab0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:01.617156 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e20023-94c2-422b-9056-4696c16eab0c admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:01.617437 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0247dd73-4818-4107-8e33-0a4051cfb392 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:01.626121 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92378359-59ff-4783-b6c4-4fb29c9b71b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:01.627597 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92378359-59ff-4783-b6c4-4fb29c9b71b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:01.628078 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92378359-59ff-4783-b6c4-4fb29c9b71b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:01.628487 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92378359-59ff-4783-b6c4-4fb29c9b71b8 admin admin] PATH_INFO: `/v3/users/67f34090b91d4b0cb5f7bd6f8db47de2` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:01.630943 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92378359-59ff-4783-b6c4-4fb29c9b71b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67f34090b91d4b0cb5f7bd6f8db47de2)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:01.632059 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0284572e-e47d-4c87-b9a5-d845a1f5240b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:01.634914 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92378359-59ff-4783-b6c4-4fb29c9b71b8 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:01.635966 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0284572e-e47d-4c87-b9a5-d845a1f5240b admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:01.635966 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0284572e-e47d-4c87-b9a5-d845a1f5240b admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:01.635966 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0284572e-e47d-4c87-b9a5-d845a1f5240b admin admin] PATH_INFO: `/v3/projects/6c4cf8cb393a4a289118d9555559f342/users/6c1e1e06642e43c291091b0a9941edf7/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:01.651863 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:01.663458 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0284572e-e47d-4c87-b9a5-d845a1f5240b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c4cf8cb393a4a289118d9555559f342, user_id=6c1e1e06642e43c291091b0a9941edf7, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:01.669626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0284572e-e47d-4c87-b9a5-d845a1f5240b admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:01.716564 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1026/2025] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:31:01 2022] PUT /identity/v3/projects/6c4cf8cb393a4a289118d9555559f342/users/6c1e1e06642e43c291091b0a9941edf7/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:01.741830 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c96ab1-4070-4ba4-82d7-c80f80cfd3a0 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:01.743565 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c96ab1-4070-4ba4-82d7-c80f80cfd3a0 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:01.744582 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c96ab1-4070-4ba4-82d7-c80f80cfd3a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:01.766801 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-92378359-59ff-4783-b6c4-4fb29c9b71b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67f34090b91d4b0cb5f7bd6f8db47de2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:01.796438 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-92378359-59ff-4783-b6c4-4fb29c9b71b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67f34090b91d4b0cb5f7bd6f8db47de2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:01.797445 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-90c96ab1-4070-4ba4-82d7-c80f80cfd3a0 None None] MFA Rules not processed for user `6c1e1e06642e43c291091b0a9941edf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:01.797752 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-92378359-59ff-4783-b6c4-4fb29c9b71b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67f34090b91d4b0cb5f7bd6f8db47de2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:01.799029 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-92378359-59ff-4783-b6c4-4fb29c9b71b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67f34090b91d4b0cb5f7bd6f8db47de2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:01.799459 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-90c96ab1-4070-4ba4-82d7-c80f80cfd3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:01.837694 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-92378359-59ff-4783-b6c4-4fb29c9b71b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67f34090b91d4b0cb5f7bd6f8db47de2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:01.846713 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:01.901759 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1000/2026] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:31:01 2022] DELETE /identity/v3/users/67f34090b91d4b0cb5f7bd6f8db47de2 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:01.908646 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce9908-d25f-499a-97b4-b3b8ade21425 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:01.909203 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce9908-d25f-499a-97b4-b3b8ade21425 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:01.909742 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce9908-d25f-499a-97b4-b3b8ade21425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:01.945672 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-52ce9908-d25f-499a-97b4-b3b8ade21425 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:01.947968 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-52ce9908-d25f-499a-97b4-b3b8ade21425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:01.955006 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1027/2027] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:31:01.964917 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c96ab1-4070-4ba4-82d7-c80f80cfd3a0 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:01.972425 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-90c96ab1-4070-4ba4-82d7-c80f80cfd3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:02.095749 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1001/2028] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:31:02.109517 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c96ab1-4070-4ba4-82d7-c80f80cfd3a0 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:02.112488 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ce9908-d25f-499a-97b4-b3b8ade21425 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:02.115974 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-52ce9908-d25f-499a-97b4-b3b8ade21425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:02.118596 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1692f778-e69f-4d1e-86eb-44250fa4acae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:02.119764 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692f778-e69f-4d1e-86eb-44250fa4acae admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:02.120266 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692f778-e69f-4d1e-86eb-44250fa4acae admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:02.120753 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692f778-e69f-4d1e-86eb-44250fa4acae admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:02.122220 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1692f778-e69f-4d1e-86eb-44250fa4acae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:02.127687 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1692f778-e69f-4d1e-86eb-44250fa4acae admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:02.201319 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1028/2029] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:31:01 2022] POST /identity/v3/projects => generated 389 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:31:02.202898 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1692f778-e69f-4d1e-86eb-44250fa4acae admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:02.255789 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ce9908-d25f-499a-97b4-b3b8ade21425 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:02.355432 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1692f778-e69f-4d1e-86eb-44250fa4acae admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:02.367179 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81ca12-be3c-403d-a1d5-7cfac12ae1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:02.368575 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81ca12-be3c-403d-a1d5-7cfac12ae1da admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:02.369064 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81ca12-be3c-403d-a1d5-7cfac12ae1da admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:02.369767 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81ca12-be3c-403d-a1d5-7cfac12ae1da admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:02.371062 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce81ca12-be3c-403d-a1d5-7cfac12ae1da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:02.375351 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4175bdee-fd14-476d-a2d3-0ee0587fb95b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:02.376542 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175bdee-fd14-476d-a2d3-0ee0587fb95b service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:02.377006 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175bdee-fd14-476d-a2d3-0ee0587fb95b service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:02.377672 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175bdee-fd14-476d-a2d3-0ee0587fb95b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:02.378811 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce81ca12-be3c-403d-a1d5-7cfac12ae1da admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:02.401172 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:02.404253 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4175bdee-fd14-476d-a2d3-0ee0587fb95b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:02.407838 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4175bdee-fd14-476d-a2d3-0ee0587fb95b service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:02.442893 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1029/2030] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:31:02 2022] POST /identity/v3/users => generated 301 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:31:02.445128 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1002/2031] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:31:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:31:02.447514 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81ca12-be3c-403d-a1d5-7cfac12ae1da admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:02.472696 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81ca12-be3c-403d-a1d5-7cfac12ae1da admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:02.482618 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d5296d-4ba3-447e-8b52-10a55e0964c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:02.484646 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5296d-4ba3-447e-8b52-10a55e0964c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:02.485897 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5296d-4ba3-447e-8b52-10a55e0964c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:02.486600 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5296d-4ba3-447e-8b52-10a55e0964c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:02.488242 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d5296d-4ba3-447e-8b52-10a55e0964c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:02.491956 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d5296d-4ba3-447e-8b52-10a55e0964c4 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:02.522392 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1030/2032] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:31:02 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:31:02.522689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4175bdee-fd14-476d-a2d3-0ee0587fb95b service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:02.549187 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4175bdee-fd14-476d-a2d3-0ee0587fb95b service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:02.560675 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4210f47-63f1-4aa3-b02d-f499f90d757e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:02.567740 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4210f47-63f1-4aa3-b02d-f499f90d757e admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:02.568611 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4210f47-63f1-4aa3-b02d-f499f90d757e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:02.569949 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4210f47-63f1-4aa3-b02d-f499f90d757e admin admin] PATH_INFO: `/v3/projects/d2c46ec851924275a1a990c997fa2161/users/7b6cbe53394e49b2ac602e6f711deedd/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:02.631433 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4210f47-63f1-4aa3-b02d-f499f90d757e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2c46ec851924275a1a990c997fa2161, user_id=7b6cbe53394e49b2ac602e6f711deedd, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:02.637742 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4210f47-63f1-4aa3-b02d-f499f90d757e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:02.680515 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1003/2033] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:31:02 2022] PUT /identity/v3/projects/d2c46ec851924275a1a990c997fa2161/users/7b6cbe53394e49b2ac602e6f711deedd/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:31:02.683224 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d5296d-4ba3-447e-8b52-10a55e0964c4 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:02.685357 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4210f47-63f1-4aa3-b02d-f499f90d757e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:02.689140 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e4210f47-63f1-4aa3-b02d-f499f90d757e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:02.722257 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d5296d-4ba3-447e-8b52-10a55e0964c4 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:02.777242 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:INSERT=2,keystone:DELETE=1,keystone:SELECT=97 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:02.855539 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4210f47-63f1-4aa3-b02d-f499f90d757e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:02.866169 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a97265-2516-47ab-806d-2cbd5d27ac91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:02.868716 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a97265-2516-47ab-806d-2cbd5d27ac91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:02.869764 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a97265-2516-47ab-806d-2cbd5d27ac91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:02.870717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a97265-2516-47ab-806d-2cbd5d27ac91 admin admin] PATH_INFO: `/v3/users/bc964b017d2f44bbbcaf06684b213e23` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:02.874426 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a97265-2516-47ab-806d-2cbd5d27ac91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc964b017d2f44bbbcaf06684b213e23)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:02.879576 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a97265-2516-47ab-806d-2cbd5d27ac91 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:02.993176 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f6e4e4-16ad-4083-9316-5780161b110a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:02.995915 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6e4e4-16ad-4083-9316-5780161b110a admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:02.996805 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6e4e4-16ad-4083-9316-5780161b110a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:02.997376 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6e4e4-16ad-4083-9316-5780161b110a admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:02.999020 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f6e4e4-16ad-4083-9316-5780161b110a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:03.002636 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f6e4e4-16ad-4083-9316-5780161b110a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:03.030754 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1031/2034] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:31:02 2022] GET /identity/v3/roles => generated 1420 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:31:03.038505 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f6e4e4-16ad-4083-9316-5780161b110a admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:03.052397 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e7a97265-2516-47ab-806d-2cbd5d27ac91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc964b017d2f44bbbcaf06684b213e23'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:03.062384 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f6e4e4-16ad-4083-9316-5780161b110a admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:03.071674 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a536a5ff-7033-4d38-908d-fbdc934af971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:03.073051 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e7a97265-2516-47ab-806d-2cbd5d27ac91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc964b017d2f44bbbcaf06684b213e23'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:03.073555 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a536a5ff-7033-4d38-908d-fbdc934af971 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:03.074215 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a536a5ff-7033-4d38-908d-fbdc934af971 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:03.074834 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a536a5ff-7033-4d38-908d-fbdc934af971 admin admin] PATH_INFO: `/v3/projects/d2c46ec851924275a1a990c997fa2161/users/7b6cbe53394e49b2ac602e6f711deedd/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:03.075156 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e7a97265-2516-47ab-806d-2cbd5d27ac91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc964b017d2f44bbbcaf06684b213e23'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:03.076255 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e7a97265-2516-47ab-806d-2cbd5d27ac91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc964b017d2f44bbbcaf06684b213e23'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:03.081018 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a536a5ff-7033-4d38-908d-fbdc934af971 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2c46ec851924275a1a990c997fa2161, user_id=7b6cbe53394e49b2ac602e6f711deedd, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:03.096135 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a536a5ff-7033-4d38-908d-fbdc934af971 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:03.102096 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e7a97265-2516-47ab-806d-2cbd5d27ac91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc964b017d2f44bbbcaf06684b213e23'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:03.127657 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1004/2036] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:31:02 2022] DELETE /identity/v3/users/bc964b017d2f44bbbcaf06684b213e23 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:03.129920 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1032/2036] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:31:03 2022] PUT /identity/v3/projects/d2c46ec851924275a1a990c997fa2161/users/7b6cbe53394e49b2ac602e6f711deedd/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:03.148047 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb25e12-3800-4493-b418-1f66b13e1e0e None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:03.148449 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb25e12-3800-4493-b418-1f66b13e1e0e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:03.148921 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb25e12-3800-4493-b418-1f66b13e1e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:03.151616 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe222094-72cf-4469-8de9-2e3029df0a66 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:03.152696 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe222094-72cf-4469-8de9-2e3029df0a66 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:03.155182 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe222094-72cf-4469-8de9-2e3029df0a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:03.187728 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-2cb25e12-3800-4493-b418-1f66b13e1e0e None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:03.192603 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2cb25e12-3800-4493-b418-1f66b13e1e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:03.258024 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-fe222094-72cf-4469-8de9-2e3029df0a66 None None] MFA Rules not processed for user `7b6cbe53394e49b2ac602e6f711deedd`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:03.262409 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-fe222094-72cf-4469-8de9-2e3029df0a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:03.327780 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1005/2037] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:31:03.363891 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb25e12-3800-4493-b418-1f66b13e1e0e None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:03.367900 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2cb25e12-3800-4493-b418-1f66b13e1e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:03.449225 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1033/2038] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:03 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:31:03.470221 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe222094-72cf-4469-8de9-2e3029df0a66 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:03.472874 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-fe222094-72cf-4469-8de9-2e3029df0a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:03.502606 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:03.528785 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb25e12-3800-4493-b418-1f66b13e1e0e None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:03.543416 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8131c0-33ed-427c-ab0c-1e74d1762817 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:03.545735 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8131c0-33ed-427c-ab0c-1e74d1762817 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:03.549082 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8131c0-33ed-427c-ab0c-1e74d1762817 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:03.549957 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8131c0-33ed-427c-ab0c-1e74d1762817 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:03.576632 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8131c0-33ed-427c-ab0c-1e74d1762817 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:03.580375 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8131c0-33ed-427c-ab0c-1e74d1762817 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:03.594434 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe222094-72cf-4469-8de9-2e3029df0a66 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:03.601359 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f37afd8-c773-4c7f-a445-f106aba1a0fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:03.602801 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f37afd8-c773-4c7f-a445-f106aba1a0fc service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:03.603959 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f37afd8-c773-4c7f-a445-f106aba1a0fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:03.604632 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f37afd8-c773-4c7f-a445-f106aba1a0fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:03.611414 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1006/2039] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:31:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:31:03.615125 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:03.623782 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f37afd8-c773-4c7f-a445-f106aba1a0fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:03.628907 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f37afd8-c773-4c7f-a445-f106aba1a0fc service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:03.646079 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1034/2040] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:31:03 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:31:03.886062 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8131c0-33ed-427c-ab0c-1e74d1762817 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:03.889696 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fa8131c0-33ed-427c-ab0c-1e74d1762817 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:03.911513 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8131c0-33ed-427c-ab0c-1e74d1762817 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:03.924088 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45f39f6-019e-47c5-ac4f-77ff2e058dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:03.930996 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f39f6-019e-47c5-ac4f-77ff2e058dfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:03.931517 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f39f6-019e-47c5-ac4f-77ff2e058dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:03.932280 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f39f6-019e-47c5-ac4f-77ff2e058dfd admin admin] PATH_INFO: `/v3/users/6c1e1e06642e43c291091b0a9941edf7` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:03.934718 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45f39f6-019e-47c5-ac4f-77ff2e058dfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c1e1e06642e43c291091b0a9941edf7)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:03.940631 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45f39f6-019e-47c5-ac4f-77ff2e058dfd admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:04.081883 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a45f39f6-019e-47c5-ac4f-77ff2e058dfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c1e1e06642e43c291091b0a9941edf7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:04.103740 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a45f39f6-019e-47c5-ac4f-77ff2e058dfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c1e1e06642e43c291091b0a9941edf7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:04.105399 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a45f39f6-019e-47c5-ac4f-77ff2e058dfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c1e1e06642e43c291091b0a9941edf7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:04.106963 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a45f39f6-019e-47c5-ac4f-77ff2e058dfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c1e1e06642e43c291091b0a9941edf7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:04.149126 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a45f39f6-019e-47c5-ac4f-77ff2e058dfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c1e1e06642e43c291091b0a9941edf7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:04.196123 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1007/2041] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:31:03 2022] DELETE /identity/v3/users/6c1e1e06642e43c291091b0a9941edf7 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:04.204412 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658cfaa9-3a4f-485f-b2b5-0ade7b5060be None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:04.205111 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658cfaa9-3a4f-485f-b2b5-0ade7b5060be None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:04.208089 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658cfaa9-3a4f-485f-b2b5-0ade7b5060be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:04.275402 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-658cfaa9-3a4f-485f-b2b5-0ade7b5060be None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:04.285438 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-658cfaa9-3a4f-485f-b2b5-0ade7b5060be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:04.436886 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45f39f6-019e-47c5-ac4f-77ff2e058dfd admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:04.441060 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a45f39f6-019e-47c5-ac4f-77ff2e058dfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:04.474424 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1035/2042] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:31:04.490498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45f39f6-019e-47c5-ac4f-77ff2e058dfd admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:04.499106 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658cfaa9-3a4f-485f-b2b5-0ade7b5060be None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:04.500487 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-658cfaa9-3a4f-485f-b2b5-0ade7b5060be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:04.504407 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9fdc56-10a0-4596-afe4-a70f2dd9d745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:04.508076 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9fdc56-10a0-4596-afe4-a70f2dd9d745 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:04.508858 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9fdc56-10a0-4596-afe4-a70f2dd9d745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:04.509343 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9fdc56-10a0-4596-afe4-a70f2dd9d745 admin admin] PATH_INFO: `/v3/projects/fc7c501a92cb43a18fea59e18066d83a` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:04.513002 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9fdc56-10a0-4596-afe4-a70f2dd9d745 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc7c501a92cb43a18fea59e18066d83a)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:04.517170 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9fdc56-10a0-4596-afe4-a70f2dd9d745 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:04.645179 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0f9fdc56-10a0-4596-afe4-a70f2dd9d745 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc7c501a92cb43a18fea59e18066d83a', 'request_id': 'req-0f9fdc56-10a0-4596-afe4-a70f2dd9d745'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:04.660292 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658cfaa9-3a4f-485f-b2b5-0ade7b5060be None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:04.661317 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0f9fdc56-10a0-4596-afe4-a70f2dd9d745 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc7c501a92cb43a18fea59e18066d83a', 'request_id': 'req-0f9fdc56-10a0-4596-afe4-a70f2dd9d745'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:04.681789 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0f9fdc56-10a0-4596-afe4-a70f2dd9d745 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc7c501a92cb43a18fea59e18066d83a', 'request_id': 'req-0f9fdc56-10a0-4596-afe4-a70f2dd9d745'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:04.714901 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0f9fdc56-10a0-4596-afe4-a70f2dd9d745 admin admin] The token cache is being invalidate because project fc7c501a92cb43a18fea59e18066d83a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:31:04.716509 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0f9fdc56-10a0-4596-afe4-a70f2dd9d745 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:04.720943 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0f9fdc56-10a0-4596-afe4-a70f2dd9d745 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:04.726444 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1008/2043] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:31:04 2022] DELETE /identity/v3/projects/fc7c501a92cb43a18fea59e18066d83a => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:04.733462 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9fdc56-10a0-4596-afe4-a70f2dd9d745 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:04.736696 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0f9fdc56-10a0-4596-afe4-a70f2dd9d745 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:04.818523 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8735324e-a1a5-4db7-a18f-ecabeb7e8e1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:04.819973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735324e-a1a5-4db7-a18f-ecabeb7e8e1d service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:04.820633 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735324e-a1a5-4db7-a18f-ecabeb7e8e1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:04.821262 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735324e-a1a5-4db7-a18f-ecabeb7e8e1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:04.825281 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8735324e-a1a5-4db7-a18f-ecabeb7e8e1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:04.898272 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:04.971792 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9fdc56-10a0-4596-afe4-a70f2dd9d745 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:04.981808 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17448205-4328-4ea1-bd73-25c2ad98b54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:04.985152 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17448205-4328-4ea1-bd73-25c2ad98b54d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:04.985609 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17448205-4328-4ea1-bd73-25c2ad98b54d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:04.986188 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17448205-4328-4ea1-bd73-25c2ad98b54d admin admin] PATH_INFO: `/v3/users/72c2735c521e452aba5189487d1ca3cc` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:04.988867 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17448205-4328-4ea1-bd73-25c2ad98b54d admin admin] RBAC: Authorizing `identity:delete_user(user_id=72c2735c521e452aba5189487d1ca3cc)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:04.993360 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17448205-4328-4ea1-bd73-25c2ad98b54d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:05.090349 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8735324e-a1a5-4db7-a18f-ecabeb7e8e1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:05.098433 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8735324e-a1a5-4db7-a18f-ecabeb7e8e1d service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:05.132816 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:05.146788 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1036/2044] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:31:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 650 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:31:05.155712 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-17448205-4328-4ea1-bd73-25c2ad98b54d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72c2735c521e452aba5189487d1ca3cc'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:05.186980 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-17448205-4328-4ea1-bd73-25c2ad98b54d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72c2735c521e452aba5189487d1ca3cc'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:05.196622 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-17448205-4328-4ea1-bd73-25c2ad98b54d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72c2735c521e452aba5189487d1ca3cc'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:05.198630 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-17448205-4328-4ea1-bd73-25c2ad98b54d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72c2735c521e452aba5189487d1ca3cc'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:05.222265 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8735324e-a1a5-4db7-a18f-ecabeb7e8e1d service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:05.225679 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8735324e-a1a5-4db7-a18f-ecabeb7e8e1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:05.301341 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8735324e-a1a5-4db7-a18f-ecabeb7e8e1d service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:05.304700 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-17448205-4328-4ea1-bd73-25c2ad98b54d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72c2735c521e452aba5189487d1ca3cc'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:05.356274 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ad863-6c86-450d-8ca4-8b499254eb4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:05.364906 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ad863-6c86-450d-8ca4-8b499254eb4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:05.366217 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ad863-6c86-450d-8ca4-8b499254eb4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:05.380179 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1009/2045] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:31:04 2022] DELETE /identity/v3/users/72c2735c521e452aba5189487d1ca3cc => generated 0 bytes in 649 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:31:05.399100 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ad863-6c86-450d-8ca4-8b499254eb4e admin admin] PATH_INFO: `/v3/projects/1c586c98abd2438e96a5f3c56c4b476d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:05.413018 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ad863-6c86-450d-8ca4-8b499254eb4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c586c98abd2438e96a5f3c56c4b476d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:05.434104 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ad863-6c86-450d-8ca4-8b499254eb4e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:05.648583 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e71ad863-6c86-450d-8ca4-8b499254eb4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c586c98abd2438e96a5f3c56c4b476d', 'request_id': 'req-e71ad863-6c86-450d-8ca4-8b499254eb4e'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:05.667137 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e71ad863-6c86-450d-8ca4-8b499254eb4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c586c98abd2438e96a5f3c56c4b476d', 'request_id': 'req-e71ad863-6c86-450d-8ca4-8b499254eb4e'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:05.689052 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e71ad863-6c86-450d-8ca4-8b499254eb4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c586c98abd2438e96a5f3c56c4b476d', 'request_id': 'req-e71ad863-6c86-450d-8ca4-8b499254eb4e'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:05.722487 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e71ad863-6c86-450d-8ca4-8b499254eb4e admin admin] The token cache is being invalidate because project 1c586c98abd2438e96a5f3c56c4b476d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:31:05.723233 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e71ad863-6c86-450d-8ca4-8b499254eb4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:05.731650 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e71ad863-6c86-450d-8ca4-8b499254eb4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:05.736205 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1037/2046] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:31:05 2022] DELETE /identity/v3/projects/1c586c98abd2438e96a5f3c56c4b476d => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:31:05.742554 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17448205-4328-4ea1-bd73-25c2ad98b54d admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:05.749210 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-17448205-4328-4ea1-bd73-25c2ad98b54d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:05.985099 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17448205-4328-4ea1-bd73-25c2ad98b54d admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:06.151549 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada0d833-02dd-4b03-9283-9a8deabdda05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:06.154043 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0d833-02dd-4b03-9283-9a8deabdda05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:06.154644 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0d833-02dd-4b03-9283-9a8deabdda05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:06.155238 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0d833-02dd-4b03-9283-9a8deabdda05 admin admin] PATH_INFO: `/v3/users/b159bd7b405246beacec59da6b5c44a6` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:06.157866 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada0d833-02dd-4b03-9283-9a8deabdda05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b159bd7b405246beacec59da6b5c44a6)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:06.162229 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada0d833-02dd-4b03-9283-9a8deabdda05 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:06.194058 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:06.295916 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ada0d833-02dd-4b03-9283-9a8deabdda05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b159bd7b405246beacec59da6b5c44a6'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:06.311049 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ada0d833-02dd-4b03-9283-9a8deabdda05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b159bd7b405246beacec59da6b5c44a6'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:06.312275 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ada0d833-02dd-4b03-9283-9a8deabdda05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b159bd7b405246beacec59da6b5c44a6'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:06.313697 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ada0d833-02dd-4b03-9283-9a8deabdda05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b159bd7b405246beacec59da6b5c44a6'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:06.339599 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ada0d833-02dd-4b03-9283-9a8deabdda05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b159bd7b405246beacec59da6b5c44a6'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:06.373071 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1010/2047] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:31:05 2022] DELETE /identity/v3/users/b159bd7b405246beacec59da6b5c44a6 => generated 0 bytes in 632 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:06.852214 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ad863-6c86-450d-8ca4-8b499254eb4e admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:06.861799 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e71ad863-6c86-450d-8ca4-8b499254eb4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:06.930015 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ad863-6c86-450d-8ca4-8b499254eb4e admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:07.104065 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670fb629-0419-4b07-98c4-8cfc1cf3bbbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:07.106482 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670fb629-0419-4b07-98c4-8cfc1cf3bbbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:07.107181 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670fb629-0419-4b07-98c4-8cfc1cf3bbbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:07.107661 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670fb629-0419-4b07-98c4-8cfc1cf3bbbc admin admin] PATH_INFO: `/v3/projects/6c4cf8cb393a4a289118d9555559f342` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:07.109525 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670fb629-0419-4b07-98c4-8cfc1cf3bbbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c4cf8cb393a4a289118d9555559f342)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:07.112586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670fb629-0419-4b07-98c4-8cfc1cf3bbbc admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:07.195250 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-670fb629-0419-4b07-98c4-8cfc1cf3bbbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c4cf8cb393a4a289118d9555559f342', 'request_id': 'req-670fb629-0419-4b07-98c4-8cfc1cf3bbbc'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:07.202495 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-670fb629-0419-4b07-98c4-8cfc1cf3bbbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c4cf8cb393a4a289118d9555559f342', 'request_id': 'req-670fb629-0419-4b07-98c4-8cfc1cf3bbbc'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:07.217290 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-670fb629-0419-4b07-98c4-8cfc1cf3bbbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c4cf8cb393a4a289118d9555559f342', 'request_id': 'req-670fb629-0419-4b07-98c4-8cfc1cf3bbbc'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:07.242489 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-670fb629-0419-4b07-98c4-8cfc1cf3bbbc admin admin] The token cache is being invalidate because project 6c4cf8cb393a4a289118d9555559f342 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:31:07.242934 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-670fb629-0419-4b07-98c4-8cfc1cf3bbbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:07.243827 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-670fb629-0419-4b07-98c4-8cfc1cf3bbbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:07.246626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1038/2048] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:31:06 2022] DELETE /identity/v3/projects/6c4cf8cb393a4a289118d9555559f342 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:07.254513 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada0d833-02dd-4b03-9283-9a8deabdda05 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:07.257701 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ada0d833-02dd-4b03-9283-9a8deabdda05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:07.282202 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada0d833-02dd-4b03-9283-9a8deabdda05 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:07.412608 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abe843d-ee86-4fb8-942e-a9c074446dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:07.414230 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe843d-ee86-4fb8-942e-a9c074446dc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:07.414644 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe843d-ee86-4fb8-942e-a9c074446dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:07.415052 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe843d-ee86-4fb8-942e-a9c074446dc6 admin admin] PATH_INFO: `/v3/users/7b6cbe53394e49b2ac602e6f711deedd` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:07.417221 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abe843d-ee86-4fb8-942e-a9c074446dc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b6cbe53394e49b2ac602e6f711deedd)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:07.420938 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abe843d-ee86-4fb8-942e-a9c074446dc6 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:07.511203 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670fb629-0419-4b07-98c4-8cfc1cf3bbbc admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:07.514896 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-670fb629-0419-4b07-98c4-8cfc1cf3bbbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:07.519116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6abe843d-ee86-4fb8-942e-a9c074446dc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b6cbe53394e49b2ac602e6f711deedd'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:07.530385 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6abe843d-ee86-4fb8-942e-a9c074446dc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b6cbe53394e49b2ac602e6f711deedd'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:07.531285 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6abe843d-ee86-4fb8-942e-a9c074446dc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b6cbe53394e49b2ac602e6f711deedd'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:07.531976 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6abe843d-ee86-4fb8-942e-a9c074446dc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b6cbe53394e49b2ac602e6f711deedd'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:07.551926 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6abe843d-ee86-4fb8-942e-a9c074446dc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b6cbe53394e49b2ac602e6f711deedd'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:07.580195 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1011/2049] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:31:07 2022] DELETE /identity/v3/users/7b6cbe53394e49b2ac602e6f711deedd => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:07.607359 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670fb629-0419-4b07-98c4-8cfc1cf3bbbc admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:07.632232 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:07.754205 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abe843d-ee86-4fb8-942e-a9c074446dc6 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:07.760147 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6abe843d-ee86-4fb8-942e-a9c074446dc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:07.800887 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:07.825875 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c606609-fa21-4861-b46d-f728eac3d895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:07.833467 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c606609-fa21-4861-b46d-f728eac3d895 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:07.835131 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c606609-fa21-4861-b46d-f728eac3d895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:07.836351 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c606609-fa21-4861-b46d-f728eac3d895 admin admin] PATH_INFO: `/v3/projects/65d6599fa1f546239055f1d44f2ae644` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:07.840033 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c606609-fa21-4861-b46d-f728eac3d895 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65d6599fa1f546239055f1d44f2ae644)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:07.847514 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c606609-fa21-4861-b46d-f728eac3d895 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:08.058969 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abe843d-ee86-4fb8-942e-a9c074446dc6 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:08.064795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0c606609-fa21-4861-b46d-f728eac3d895 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65d6599fa1f546239055f1d44f2ae644', 'request_id': 'req-0c606609-fa21-4861-b46d-f728eac3d895'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:08.098069 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0c606609-fa21-4861-b46d-f728eac3d895 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65d6599fa1f546239055f1d44f2ae644', 'request_id': 'req-0c606609-fa21-4861-b46d-f728eac3d895'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:08.125752 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0c606609-fa21-4861-b46d-f728eac3d895 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65d6599fa1f546239055f1d44f2ae644', 'request_id': 'req-0c606609-fa21-4861-b46d-f728eac3d895'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:08.175635 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0c606609-fa21-4861-b46d-f728eac3d895 admin admin] The token cache is being invalidate because project 65d6599fa1f546239055f1d44f2ae644 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:31:08.176603 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0c606609-fa21-4861-b46d-f728eac3d895 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:08.178052 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0c606609-fa21-4861-b46d-f728eac3d895 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:08.187257 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1039/2050] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:31:07 2022] DELETE /identity/v3/projects/65d6599fa1f546239055f1d44f2ae644 => generated 0 bytes in 672 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:08.190454 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33904d2a-edcd-468d-9091-8ab0f1c03868 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:08.190979 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33904d2a-edcd-468d-9091-8ab0f1c03868 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:08.191503 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33904d2a-edcd-468d-9091-8ab0f1c03868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:08.269421 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-33904d2a-edcd-468d-9091-8ab0f1c03868 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:08.291596 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-33904d2a-edcd-468d-9091-8ab0f1c03868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:08.317438 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cde5c7-45d8-440e-b2dd-173b8818c1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:08.323060 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cde5c7-45d8-440e-b2dd-173b8818c1ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:08.325181 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cde5c7-45d8-440e-b2dd-173b8818c1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:08.325181 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cde5c7-45d8-440e-b2dd-173b8818c1ba admin admin] PATH_INFO: `/v3/projects/702d9d1b50ab480e9604807eebed8182` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:08.338128 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cde5c7-45d8-440e-b2dd-173b8818c1ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=702d9d1b50ab480e9604807eebed8182)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:08.353798 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cde5c7-45d8-440e-b2dd-173b8818c1ba admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:08.357534 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1040/2051] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:31:08.360959 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd107a-303a-40c9-8d89-31d8ecc56ba2 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:08.361786 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd107a-303a-40c9-8d89-31d8ecc56ba2 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:08.362546 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd107a-303a-40c9-8d89-31d8ecc56ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:08.437579 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-49cd107a-303a-40c9-8d89-31d8ecc56ba2 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:08.445349 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-49cd107a-303a-40c9-8d89-31d8ecc56ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:08.475686 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1041/2052] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:31:08.480866 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd107a-303a-40c9-8d89-31d8ecc56ba2 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:08.510436 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd107a-303a-40c9-8d89-31d8ecc56ba2 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:08.518154 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-68cde5c7-45d8-440e-b2dd-173b8818c1ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '702d9d1b50ab480e9604807eebed8182', 'request_id': 'req-68cde5c7-45d8-440e-b2dd-173b8818c1ba'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:08.534126 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-68cde5c7-45d8-440e-b2dd-173b8818c1ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '702d9d1b50ab480e9604807eebed8182', 'request_id': 'req-68cde5c7-45d8-440e-b2dd-173b8818c1ba'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:08.550024 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-68cde5c7-45d8-440e-b2dd-173b8818c1ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '702d9d1b50ab480e9604807eebed8182', 'request_id': 'req-68cde5c7-45d8-440e-b2dd-173b8818c1ba'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:08.597156 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-68cde5c7-45d8-440e-b2dd-173b8818c1ba admin admin] The token cache is being invalidate because project 702d9d1b50ab480e9604807eebed8182 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:31:08.597385 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-68cde5c7-45d8-440e-b2dd-173b8818c1ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:08.598754 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-68cde5c7-45d8-440e-b2dd-173b8818c1ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:08.607354 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1012/2053] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:31:07 2022] DELETE /identity/v3/projects/702d9d1b50ab480e9604807eebed8182 => generated 0 bytes in 855 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:08.616983 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cde5c7-45d8-440e-b2dd-173b8818c1ba admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:08.626869 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-68cde5c7-45d8-440e-b2dd-173b8818c1ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:08.739563 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b91a3-cd2e-4527-a283-1f9a537af67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:08.741489 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b91a3-cd2e-4527-a283-1f9a537af67c admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:08.742045 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b91a3-cd2e-4527-a283-1f9a537af67c admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:08.742607 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b91a3-cd2e-4527-a283-1f9a537af67c admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:08.744807 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108b91a3-cd2e-4527-a283-1f9a537af67c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:31:08.745550 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108b91a3-cd2e-4527-a283-1f9a537af67c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:08.750295 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108b91a3-cd2e-4527-a283-1f9a537af67c admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:08.814616 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1042/2054] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:31:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:31:08.823803 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b91a3-cd2e-4527-a283-1f9a537af67c admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:08.831165 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:08.831558 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-108b91a3-cd2e-4527-a283-1f9a537af67c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:08.908720 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b91a3-cd2e-4527-a283-1f9a537af67c admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:08.918350 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cde5c7-45d8-440e-b2dd-173b8818c1ba admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:08.939153 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a9518f-5fe1-4f26-8d13-bed12c001fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:08.943108 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9518f-5fe1-4f26-8d13-bed12c001fe8 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:08.945928 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9518f-5fe1-4f26-8d13-bed12c001fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:08.948420 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9518f-5fe1-4f26-8d13-bed12c001fe8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:08.950934 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c145b-16b9-4c59-b662-eaa48625af8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:08.952205 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a9518f-5fe1-4f26-8d13-bed12c001fe8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:08.952409 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c145b-16b9-4c59-b662-eaa48625af8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:08.952857 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c145b-16b9-4c59-b662-eaa48625af8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:08.953179 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c145b-16b9-4c59-b662-eaa48625af8a admin admin] PATH_INFO: `/v3/users/d9c32b0a455c4382b2ff86d023238077` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:08.956238 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260c145b-16b9-4c59-b662-eaa48625af8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9c32b0a455c4382b2ff86d023238077)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:08.960632 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a9518f-5fe1-4f26-8d13-bed12c001fe8 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:08.962304 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260c145b-16b9-4c59-b662-eaa48625af8a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:09.073716 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1043/2055] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:31:08 2022] POST /identity/v3/projects => generated 381 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:31:09.078291 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a9518f-5fe1-4f26-8d13-bed12c001fe8 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:09.090111 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:09.103378 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-260c145b-16b9-4c59-b662-eaa48625af8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9c32b0a455c4382b2ff86d023238077'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:09.124049 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-260c145b-16b9-4c59-b662-eaa48625af8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9c32b0a455c4382b2ff86d023238077'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:09.124349 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-260c145b-16b9-4c59-b662-eaa48625af8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9c32b0a455c4382b2ff86d023238077'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:09.125044 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-260c145b-16b9-4c59-b662-eaa48625af8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9c32b0a455c4382b2ff86d023238077'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:09.172181 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-260c145b-16b9-4c59-b662-eaa48625af8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9c32b0a455c4382b2ff86d023238077'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:09.212489 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1013/2056] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:31:08 2022] DELETE /identity/v3/users/d9c32b0a455c4382b2ff86d023238077 => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:09.356566 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a9518f-5fe1-4f26-8d13-bed12c001fe8 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:09.551456 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8225f-57db-4e42-b0c2-d090145a2587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:09.552261 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8225f-57db-4e42-b0c2-d090145a2587 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:09.555124 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8225f-57db-4e42-b0c2-d090145a2587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:09.555124 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8225f-57db-4e42-b0c2-d090145a2587 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:09.555517 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c8225f-57db-4e42-b0c2-d090145a2587 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:09.557969 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c8225f-57db-4e42-b0c2-d090145a2587 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:09.594328 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c145b-16b9-4c59-b662-eaa48625af8a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:09.599461 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-260c145b-16b9-4c59-b662-eaa48625af8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:09.649410 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1044/2057] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:31:09 2022] POST /identity/v3/users => generated 291 bytes in 572 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:31:09.651129 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8225f-57db-4e42-b0c2-d090145a2587 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:09.651658 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c145b-16b9-4c59-b662-eaa48625af8a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:09.655035 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-56c8225f-57db-4e42-b0c2-d090145a2587 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:09.673054 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f87ec03-fac2-4e07-a9ef-7140f2711f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:09.675100 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87ec03-fac2-4e07-a9ef-7140f2711f37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:09.675863 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87ec03-fac2-4e07-a9ef-7140f2711f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:09.676630 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87ec03-fac2-4e07-a9ef-7140f2711f37 admin admin] PATH_INFO: `/v3/projects/d2c46ec851924275a1a990c997fa2161` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:09.679669 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f87ec03-fac2-4e07-a9ef-7140f2711f37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2c46ec851924275a1a990c997fa2161)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:09.681871 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8225f-57db-4e42-b0c2-d090145a2587 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:09.684612 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f87ec03-fac2-4e07-a9ef-7140f2711f37 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:09.691470 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aafd9a-a0ce-4ff3-a1f1-82ac46c89d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:09.692762 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aafd9a-a0ce-4ff3-a1f1-82ac46c89d54 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:09.693354 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aafd9a-a0ce-4ff3-a1f1-82ac46c89d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:09.694061 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aafd9a-a0ce-4ff3-a1f1-82ac46c89d54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:09.695657 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72aafd9a-a0ce-4ff3-a1f1-82ac46c89d54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:09.698595 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72aafd9a-a0ce-4ff3-a1f1-82ac46c89d54 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:09.726732 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1045/2058] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:31:09 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:31:09.734177 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aafd9a-a0ce-4ff3-a1f1-82ac46c89d54 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:09.756798 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aafd9a-a0ce-4ff3-a1f1-82ac46c89d54 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:09.770762 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adcea1c-fe5a-42f0-be2a-3bb7c0fd4ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:09.771933 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcea1c-fe5a-42f0-be2a-3bb7c0fd4ced admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:09.772544 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcea1c-fe5a-42f0-be2a-3bb7c0fd4ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:09.774142 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcea1c-fe5a-42f0-be2a-3bb7c0fd4ced admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:09.775725 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adcea1c-fe5a-42f0-be2a-3bb7c0fd4ced admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:09.780890 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adcea1c-fe5a-42f0-be2a-3bb7c0fd4ced admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:09.825259 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1046/2059] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:31:09 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:31:09.827361 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adcea1c-fe5a-42f0-be2a-3bb7c0fd4ced admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:09.839066 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-1f87ec03-fac2-4e07-a9ef-7140f2711f37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2c46ec851924275a1a990c997fa2161', 'request_id': 'req-1f87ec03-fac2-4e07-a9ef-7140f2711f37'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:09.856406 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adcea1c-fe5a-42f0-be2a-3bb7c0fd4ced admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:09.861629 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-1f87ec03-fac2-4e07-a9ef-7140f2711f37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2c46ec851924275a1a990c997fa2161', 'request_id': 'req-1f87ec03-fac2-4e07-a9ef-7140f2711f37'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:09.879129 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-1f87ec03-fac2-4e07-a9ef-7140f2711f37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2c46ec851924275a1a990c997fa2161', 'request_id': 'req-1f87ec03-fac2-4e07-a9ef-7140f2711f37'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:09.898009 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-1f87ec03-fac2-4e07-a9ef-7140f2711f37 admin admin] The token cache is being invalidate because project d2c46ec851924275a1a990c997fa2161 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:31:09.898410 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-1f87ec03-fac2-4e07-a9ef-7140f2711f37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:09.899646 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-1f87ec03-fac2-4e07-a9ef-7140f2711f37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:09.901755 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1014/2060] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:31:09 2022] DELETE /identity/v3/projects/d2c46ec851924275a1a990c997fa2161 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:09.908816 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247def5c-d973-4f93-a653-8232ceb24879 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:09.909406 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247def5c-d973-4f93-a653-8232ceb24879 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:09.910349 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247def5c-d973-4f93-a653-8232ceb24879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:09.946740 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-247def5c-d973-4f93-a653-8232ceb24879 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:09.948618 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-247def5c-d973-4f93-a653-8232ceb24879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:10.015461 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f05f2a-385e-40f8-a113-99b6bc0ca640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:10.017482 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f05f2a-385e-40f8-a113-99b6bc0ca640 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:10.017943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f05f2a-385e-40f8-a113-99b6bc0ca640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:10.018137 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f05f2a-385e-40f8-a113-99b6bc0ca640 admin admin] PATH_INFO: `/v3/projects/55c4efeea2f54994b6f45b5cc346c43b/users/ec179985fc3b41d188ca69d9e03263d0/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:10.029448 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:10.061407 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1015/2061] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:31:10.070569 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b1ca4-b5da-48c1-af9b-8ee1bf0a6c76 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:10.073967 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b1ca4-b5da-48c1-af9b-8ee1bf0a6c76 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:10.074750 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b1ca4-b5da-48c1-af9b-8ee1bf0a6c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:10.075965 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f05f2a-385e-40f8-a113-99b6bc0ca640 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55c4efeea2f54994b6f45b5cc346c43b, user_id=ec179985fc3b41d188ca69d9e03263d0, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:10.079351 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f05f2a-385e-40f8-a113-99b6bc0ca640 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:10.094200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:10.111263 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1047/2062] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:31:09 2022] PUT /identity/v3/projects/55c4efeea2f54994b6f45b5cc346c43b/users/ec179985fc3b41d188ca69d9e03263d0/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:10.114530 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e882b-ca26-42b1-94de-fbd3f392f24e None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:10.115641 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e882b-ca26-42b1-94de-fbd3f392f24e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:10.116260 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e882b-ca26-42b1-94de-fbd3f392f24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:10.153103 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-584b1ca4-b5da-48c1-af9b-8ee1bf0a6c76 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:10.158546 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-584b1ca4-b5da-48c1-af9b-8ee1bf0a6c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:10.196574 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-c33e882b-ca26-42b1-94de-fbd3f392f24e None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:10.198829 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c33e882b-ca26-42b1-94de-fbd3f392f24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:10.419345 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1048/2063] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:31:10.429012 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1016/2064] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:31:10.440774 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33e882b-ca26-42b1-94de-fbd3f392f24e None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:10.448450 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584b1ca4-b5da-48c1-af9b-8ee1bf0a6c76 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:10.452200 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-584b1ca4-b5da-48c1-af9b-8ee1bf0a6c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:10.472829 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33e882b-ca26-42b1-94de-fbd3f392f24e None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:10.494412 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f861c542-39c8-4698-9925-60b6d2ca4bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:10.494927 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861c542-39c8-4698-9925-60b6d2ca4bb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:10.495318 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861c542-39c8-4698-9925-60b6d2ca4bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:10.495767 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861c542-39c8-4698-9925-60b6d2ca4bb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:10.496655 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f861c542-39c8-4698-9925-60b6d2ca4bb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:31:10.498784 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f861c542-39c8-4698-9925-60b6d2ca4bb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:10.503249 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f861c542-39c8-4698-9925-60b6d2ca4bb0 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:10.561810 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1049/2065] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:31:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:31:10.563690 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f861c542-39c8-4698-9925-60b6d2ca4bb0 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:10.573398 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f861c542-39c8-4698-9925-60b6d2ca4bb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:10.610891 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f861c542-39c8-4698-9925-60b6d2ca4bb0 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:10.617838 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271ac5df-3815-410e-bd5e-367800cb97c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:10.619582 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ac5df-3815-410e-bd5e-367800cb97c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:10.620816 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ac5df-3815-410e-bd5e-367800cb97c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:10.620816 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ac5df-3815-410e-bd5e-367800cb97c2 admin admin] PATH_INFO: `/v3/projects/73fcc61fd03d4831a1d5e137e4b229b7` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:10.623301 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271ac5df-3815-410e-bd5e-367800cb97c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73fcc61fd03d4831a1d5e137e4b229b7)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:10.627637 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271ac5df-3815-410e-bd5e-367800cb97c2 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:10.658227 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584b1ca4-b5da-48c1-af9b-8ee1bf0a6c76 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:10.670940 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c893e1f-39a8-4b16-838c-645e730c239d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:10.671523 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c893e1f-39a8-4b16-838c-645e730c239d service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:10.671995 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c893e1f-39a8-4b16-838c-645e730c239d service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:10.672446 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c893e1f-39a8-4b16-838c-645e730c239d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:10.696874 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c893e1f-39a8-4b16-838c-645e730c239d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:10.699328 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c893e1f-39a8-4b16-838c-645e730c239d service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:10.714621 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1017/2066] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:31:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:31:10.715799 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c893e1f-39a8-4b16-838c-645e730c239d service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:10.728497 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c893e1f-39a8-4b16-838c-645e730c239d service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:10.748462 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-271ac5df-3815-410e-bd5e-367800cb97c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73fcc61fd03d4831a1d5e137e4b229b7', 'request_id': 'req-271ac5df-3815-410e-bd5e-367800cb97c2'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:10.762825 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-271ac5df-3815-410e-bd5e-367800cb97c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73fcc61fd03d4831a1d5e137e4b229b7', 'request_id': 'req-271ac5df-3815-410e-bd5e-367800cb97c2'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:10.783679 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-271ac5df-3815-410e-bd5e-367800cb97c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73fcc61fd03d4831a1d5e137e4b229b7', 'request_id': 'req-271ac5df-3815-410e-bd5e-367800cb97c2'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:10.813954 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b334161-cbed-4a14-9dd0-0eafab40c557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:10.813954 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b334161-cbed-4a14-9dd0-0eafab40c557 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:10.813954 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b334161-cbed-4a14-9dd0-0eafab40c557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:10.813954 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b334161-cbed-4a14-9dd0-0eafab40c557 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:10.814753 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b334161-cbed-4a14-9dd0-0eafab40c557 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:10.816178 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-271ac5df-3815-410e-bd5e-367800cb97c2 admin admin] The token cache is being invalidate because project 73fcc61fd03d4831a1d5e137e4b229b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:31:10.816178 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-271ac5df-3815-410e-bd5e-367800cb97c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:10.817881 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-271ac5df-3815-410e-bd5e-367800cb97c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:10.819339 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b334161-cbed-4a14-9dd0-0eafab40c557 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:10.830834 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1050/2067] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:31:10 2022] DELETE /identity/v3/projects/73fcc61fd03d4831a1d5e137e4b229b7 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:10.834570 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87106232-dc72-49aa-9baa-b13261e02b48 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:10.835284 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87106232-dc72-49aa-9baa-b13261e02b48 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:10.835831 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87106232-dc72-49aa-9baa-b13261e02b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:10.885809 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-87106232-dc72-49aa-9baa-b13261e02b48 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:10.888659 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-87106232-dc72-49aa-9baa-b13261e02b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:10.891666 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1018/2068] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:31:10 2022] POST /identity/v3/projects => generated 387 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:31:10.893367 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b334161-cbed-4a14-9dd0-0eafab40c557 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:10.897878 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9b334161-cbed-4a14-9dd0-0eafab40c557 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:10.956163 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:11.056090 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1051/2069] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:31:11.070401 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37617b14-6c89-4ec7-9a25-67848095024e None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:11.070728 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37617b14-6c89-4ec7-9a25-67848095024e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:11.070879 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37617b14-6c89-4ec7-9a25-67848095024e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:11.094300 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b334161-cbed-4a14-9dd0-0eafab40c557 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:11.103796 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d815a8-b904-4623-8cd0-8116e1c8b87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:11.104656 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d815a8-b904-4623-8cd0-8116e1c8b87a admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:11.105373 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d815a8-b904-4623-8cd0-8116e1c8b87a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:11.105777 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d815a8-b904-4623-8cd0-8116e1c8b87a admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:11.106609 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d815a8-b904-4623-8cd0-8116e1c8b87a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:11.111063 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d815a8-b904-4623-8cd0-8116e1c8b87a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:11.135097 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-37617b14-6c89-4ec7-9a25-67848095024e None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:11.144455 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-37617b14-6c89-4ec7-9a25-67848095024e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:11.150902 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1019/2070] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:31:10 2022] POST /identity/v3/users => generated 294 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:31:11.159052 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d815a8-b904-4623-8cd0-8116e1c8b87a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:11.166390 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1052/2071] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:31:11.176630 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d815a8-b904-4623-8cd0-8116e1c8b87a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:11.180926 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37617b14-6c89-4ec7-9a25-67848095024e None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:11.187396 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bcbac7-8ba0-4ed8-9bc2-97ea0dc8a53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:11.188325 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bcbac7-8ba0-4ed8-9bc2-97ea0dc8a53d admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:11.188692 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bcbac7-8ba0-4ed8-9bc2-97ea0dc8a53d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:11.188972 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bcbac7-8ba0-4ed8-9bc2-97ea0dc8a53d admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:11.189765 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bcbac7-8ba0-4ed8-9bc2-97ea0dc8a53d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:11.191734 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bcbac7-8ba0-4ed8-9bc2-97ea0dc8a53d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:11.213222 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1020/2072] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:31:11.214213 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:11.219979 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bcbac7-8ba0-4ed8-9bc2-97ea0dc8a53d admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:11.240449 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37617b14-6c89-4ec7-9a25-67848095024e None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:11.251136 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bcbac7-8ba0-4ed8-9bc2-97ea0dc8a53d admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:11.258007 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b5b4e7-2fba-4c61-9d22-63b1058cd34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:11.259266 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b5b4e7-2fba-4c61-9d22-63b1058cd34e admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:11.259266 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b5b4e7-2fba-4c61-9d22-63b1058cd34e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:11.259714 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b5b4e7-2fba-4c61-9d22-63b1058cd34e admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:11.260655 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b5b4e7-2fba-4c61-9d22-63b1058cd34e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:11.263547 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b5b4e7-2fba-4c61-9d22-63b1058cd34e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:11.263885 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f48a1-42db-40db-a47e-8e7722f393f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:11.265626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f48a1-42db-40db-a47e-8e7722f393f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:11.265626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f48a1-42db-40db-a47e-8e7722f393f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:11.266291 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f48a1-42db-40db-a47e-8e7722f393f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:11.269428 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09f48a1-42db-40db-a47e-8e7722f393f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:31:11.270023 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09f48a1-42db-40db-a47e-8e7722f393f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:11.275532 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09f48a1-42db-40db-a47e-8e7722f393f1 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:11.284681 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1021/2073] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:31:11.292958 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b5b4e7-2fba-4c61-9d22-63b1058cd34e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:11.297938 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1053/2074] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:31:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:31:11.304030 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f48a1-42db-40db-a47e-8e7722f393f1 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:11.313495 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b5b4e7-2fba-4c61-9d22-63b1058cd34e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:11.321024 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f48a1-42db-40db-a47e-8e7722f393f1 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:11.321748 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876a117-91dd-4ea8-91d7-27c8ffabbf34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:11.323378 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876a117-91dd-4ea8-91d7-27c8ffabbf34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:11.323858 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876a117-91dd-4ea8-91d7-27c8ffabbf34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:11.324349 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876a117-91dd-4ea8-91d7-27c8ffabbf34 admin admin] PATH_INFO: `/v3/projects/1a2bc96612ac4302b39f8ffe341551cb/users/3c8ea2ff51554db7b5ba525b5818ba5c/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:11.327002 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3606f6-9767-481c-9475-7bdd45906ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:11.327983 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3606f6-9767-481c-9475-7bdd45906ed3 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:11.328454 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3606f6-9767-481c-9475-7bdd45906ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:11.328780 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3606f6-9767-481c-9475-7bdd45906ed3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:11.330561 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3606f6-9767-481c-9475-7bdd45906ed3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:11.335779 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3606f6-9767-481c-9475-7bdd45906ed3 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:11.358904 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0876a117-91dd-4ea8-91d7-27c8ffabbf34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a2bc96612ac4302b39f8ffe341551cb, user_id=3c8ea2ff51554db7b5ba525b5818ba5c, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:11.364833 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0876a117-91dd-4ea8-91d7-27c8ffabbf34 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:11.396007 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1054/2076] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:31:11 2022] POST /identity/v3/projects => generated 371 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:31:11.402669 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1022/2076] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:31:11 2022] PUT /identity/v3/projects/1a2bc96612ac4302b39f8ffe341551cb/users/3c8ea2ff51554db7b5ba525b5818ba5c/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:31:11.403438 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3606f6-9767-481c-9475-7bdd45906ed3 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:11.411931 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f15c666-303d-45b9-b365-f0125d912e9f None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:11.412437 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f15c666-303d-45b9-b365-f0125d912e9f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:11.413173 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f15c666-303d-45b9-b365-f0125d912e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:11.464751 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3f15c666-303d-45b9-b365-f0125d912e9f None None] MFA Rules not processed for user `3c8ea2ff51554db7b5ba525b5818ba5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:11.468346 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3f15c666-303d-45b9-b365-f0125d912e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:11.567413 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3606f6-9767-481c-9475-7bdd45906ed3 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:11.575925 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0449f4-e9f4-491f-9542-5f574ab25773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:11.577080 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0449f4-e9f4-491f-9542-5f574ab25773 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:11.577576 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0449f4-e9f4-491f-9542-5f574ab25773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:11.578127 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0449f4-e9f4-491f-9542-5f574ab25773 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:11.579438 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0449f4-e9f4-491f-9542-5f574ab25773 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:11.583131 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0449f4-e9f4-491f-9542-5f574ab25773 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:11.627455 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1055/2077] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:31:11 2022] POST /identity/v3/users => generated 286 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:31:11.632867 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0449f4-e9f4-491f-9542-5f574ab25773 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:11.637555 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1023/2078] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:31:11.646344 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f15c666-303d-45b9-b365-f0125d912e9f None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:11.650257 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0449f4-e9f4-491f-9542-5f574ab25773 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:11.658160 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f37501-c439-490b-b845-22de7d1bef5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:11.658461 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f15c666-303d-45b9-b365-f0125d912e9f None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:11.659452 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f37501-c439-490b-b845-22de7d1bef5d admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:11.660040 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f37501-c439-490b-b845-22de7d1bef5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:11.660609 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f37501-c439-490b-b845-22de7d1bef5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:11.662102 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f37501-c439-490b-b845-22de7d1bef5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:11.665570 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f37501-c439-490b-b845-22de7d1bef5d admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:11.665880 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac8be3d-d1c1-4d82-8c67-03e8d37e1c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:11.671064 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8be3d-d1c1-4d82-8c67-03e8d37e1c4c admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:11.671516 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8be3d-d1c1-4d82-8c67-03e8d37e1c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:11.671992 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8be3d-d1c1-4d82-8c67-03e8d37e1c4c admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:11.673424 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac8be3d-d1c1-4d82-8c67-03e8d37e1c4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:11.677211 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac8be3d-d1c1-4d82-8c67-03e8d37e1c4c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:11.691242 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1056/2079] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:31:11.699911 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f37501-c439-490b-b845-22de7d1bef5d admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:11.729312 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f37501-c439-490b-b845-22de7d1bef5d admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:11.739839 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1024/2080] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:31:11 2022] POST /identity/v3/projects => generated 387 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:31:11.740140 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a75186-533c-4dcf-9e30-52532c8cfc69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:11.741218 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a75186-533c-4dcf-9e30-52532c8cfc69 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:11.741564 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a75186-533c-4dcf-9e30-52532c8cfc69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:11.742182 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a75186-533c-4dcf-9e30-52532c8cfc69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:11.743329 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a75186-533c-4dcf-9e30-52532c8cfc69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:11.746525 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a75186-533c-4dcf-9e30-52532c8cfc69 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:11.748014 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac8be3d-d1c1-4d82-8c67-03e8d37e1c4c admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:11.776923 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1057/2081] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:31:11.782396 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a75186-533c-4dcf-9e30-52532c8cfc69 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:11.800983 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:11.804976 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a75186-533c-4dcf-9e30-52532c8cfc69 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:11.884910 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac8be3d-d1c1-4d82-8c67-03e8d37e1c4c admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:11.894499 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4222277-8899-4709-971e-479ded26b74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:11.895682 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4222277-8899-4709-971e-479ded26b74f admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:11.896175 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4222277-8899-4709-971e-479ded26b74f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:11.896687 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4222277-8899-4709-971e-479ded26b74f admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:11.898090 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4222277-8899-4709-971e-479ded26b74f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:11.902175 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4222277-8899-4709-971e-479ded26b74f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:11.921326 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fbc2c2-cfb4-403d-a5aa-3af5dd1a229c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:11.922522 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fbc2c2-cfb4-403d-a5aa-3af5dd1a229c admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:11.922827 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fbc2c2-cfb4-403d-a5aa-3af5dd1a229c admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:11.923324 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fbc2c2-cfb4-403d-a5aa-3af5dd1a229c admin admin] PATH_INFO: `/v3/projects/97e3300702ff4ec9a415f1d4d3269499/users/e60703da011b4ed6b35f19f9ea4cafe7/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:11.933417 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1025/2082] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:31:11 2022] POST /identity/v3/users => generated 300 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:31:11.934753 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381abb99-a1a8-482f-9a4d-ada0a3f22c6a None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:11.935062 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381abb99-a1a8-482f-9a4d-ada0a3f22c6a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:11.935351 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381abb99-a1a8-482f-9a4d-ada0a3f22c6a None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:11.937070 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1026/2083] 10.222.0.87 () {54 vars in 921 bytes} [Tue Jun 21 18:31:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:31:11.939314 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381abb99-a1a8-482f-9a4d-ada0a3f22c6a None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:11.955534 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381abb99-a1a8-482f-9a4d-ada0a3f22c6a None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:11.964929 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fbc2c2-cfb4-403d-a5aa-3af5dd1a229c admin admin] RBAC: Authorizing `identity:create_grant(project_id=97e3300702ff4ec9a415f1d4d3269499, user_id=e60703da011b4ed6b35f19f9ea4cafe7, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:11.967454 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435c1788-38de-417a-849f-7533dcfdc3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:11.971086 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435c1788-38de-417a-849f-7533dcfdc3db admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:11.971086 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435c1788-38de-417a-849f-7533dcfdc3db admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:11.973410 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435c1788-38de-417a-849f-7533dcfdc3db admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:11.973791 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fbc2c2-cfb4-403d-a5aa-3af5dd1a229c admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:11.973960 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435c1788-38de-417a-849f-7533dcfdc3db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:11.974527 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435c1788-38de-417a-849f-7533dcfdc3db admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:12.012236 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1027/2084] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:31:12.013947 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2a38d-8472-42be-ad04-c9b55ca9c4e9 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:12.014771 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2a38d-8472-42be-ad04-c9b55ca9c4e9 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:12.015282 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2a38d-8472-42be-ad04-c9b55ca9c4e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:12.042013 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1058/2085] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:31:11 2022] PUT /identity/v3/projects/97e3300702ff4ec9a415f1d4d3269499/users/e60703da011b4ed6b35f19f9ea4cafe7/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:31:12.042013 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fbc2c2-cfb4-403d-a5aa-3af5dd1a229c admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:12.111492 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:12.213024 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-c9a2a38d-8472-42be-ad04-c9b55ca9c4e9 None None] MFA Rules not processed for user `ef1d49b36cdc4665b82487be97e9dcd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:12.215211 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c9a2a38d-8472-42be-ad04-c9b55ca9c4e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:12.258185 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fbc2c2-cfb4-403d-a5aa-3af5dd1a229c admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:12.270312 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7687d2b-8c61-41d9-a4b0-f246346e1f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:12.272631 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7687d2b-8c61-41d9-a4b0-f246346e1f6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:12.273725 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7687d2b-8c61-41d9-a4b0-f246346e1f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:12.274925 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7687d2b-8c61-41d9-a4b0-f246346e1f6c admin admin] PATH_INFO: `/v3/projects/1e1fc099cdda42c8aa5a961633e4f66a/users/df71a61377c04027b40b8f3d78b47591/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:12.307535 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7687d2b-8c61-41d9-a4b0-f246346e1f6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e1fc099cdda42c8aa5a961633e4f66a, user_id=df71a61377c04027b40b8f3d78b47591, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:12.317369 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7687d2b-8c61-41d9-a4b0-f246346e1f6c admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:12.351992 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1059/2086] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:31:12 2022] PUT /identity/v3/projects/1e1fc099cdda42c8aa5a961633e4f66a/users/df71a61377c04027b40b8f3d78b47591/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:12.354332 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0303c-c102-45ab-a7a4-bf6311c36719 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:12.355135 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0303c-c102-45ab-a7a4-bf6311c36719 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:12.355529 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0303c-c102-45ab-a7a4-bf6311c36719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:12.392414 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-34c0303c-c102-45ab-a7a4-bf6311c36719 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:12.395078 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-34c0303c-c102-45ab-a7a4-bf6311c36719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:12.499100 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1028/2087] 10.222.0.87 () {58 vars in 1018 bytes} [Tue Jun 21 18:31:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 489 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:31:12.504235 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a2a38d-8472-42be-ad04-c9b55ca9c4e9 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:12.506297 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c9a2a38d-8472-42be-ad04-c9b55ca9c4e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:12.518325 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1060/2088] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:31:12.522929 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0303c-c102-45ab-a7a4-bf6311c36719 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:12.536025 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0303c-c102-45ab-a7a4-bf6311c36719 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:12.539494 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a2a38d-8472-42be-ad04-c9b55ca9c4e9 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:12.542388 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d0ce1b-0566-46f5-a8fb-50ba4a9a65ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:12.543488 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0ce1b-0566-46f5-a8fb-50ba4a9a65ab admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:12.543949 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0ce1b-0566-46f5-a8fb-50ba4a9a65ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:12.544673 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0ce1b-0566-46f5-a8fb-50ba4a9a65ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:12.545626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d0ce1b-0566-46f5-a8fb-50ba4a9a65ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:12.547387 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d0ce1b-0566-46f5-a8fb-50ba4a9a65ab admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:12.558036 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c703317f-ca2d-438e-9aaa-c6c76c4700de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:12.559301 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c703317f-ca2d-438e-9aaa-c6c76c4700de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:12.559942 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c703317f-ca2d-438e-9aaa-c6c76c4700de admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:12.560392 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c703317f-ca2d-438e-9aaa-c6c76c4700de admin admin] PATH_INFO: `/v3/users/a03657e872e94266b12d95a602038cda` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:12.561528 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1061/2089] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:31:12.562560 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d0ce1b-0566-46f5-a8fb-50ba4a9a65ab admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:12.567878 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-04d0ce1b-0566-46f5-a8fb-50ba4a9a65ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:12.569264 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c703317f-ca2d-438e-9aaa-c6c76c4700de admin admin] RBAC: Authorizing `identity:delete_user(user_id=a03657e872e94266b12d95a602038cda)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:12.571847 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c703317f-ca2d-438e-9aaa-c6c76c4700de admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:12.702257 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c703317f-ca2d-438e-9aaa-c6c76c4700de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a03657e872e94266b12d95a602038cda'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:12.707753 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d0ce1b-0566-46f5-a8fb-50ba4a9a65ab admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:12.722805 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f282f902-7830-4a09-8453-bc41b37c86d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:12.724174 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282f902-7830-4a09-8453-bc41b37c86d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:12.724852 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282f902-7830-4a09-8453-bc41b37c86d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:12.725307 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282f902-7830-4a09-8453-bc41b37c86d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:12.740690 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c703317f-ca2d-438e-9aaa-c6c76c4700de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a03657e872e94266b12d95a602038cda'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:12.743058 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c703317f-ca2d-438e-9aaa-c6c76c4700de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a03657e872e94266b12d95a602038cda'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:12.744305 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f282f902-7830-4a09-8453-bc41b37c86d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:12.745393 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c703317f-ca2d-438e-9aaa-c6c76c4700de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a03657e872e94266b12d95a602038cda'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:12.747413 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f282f902-7830-4a09-8453-bc41b37c86d1 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:12.768232 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1062/2090] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:31:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:31:12.769768 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f282f902-7830-4a09-8453-bc41b37c86d1 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:12.778803 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:12.797341 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c703317f-ca2d-438e-9aaa-c6c76c4700de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a03657e872e94266b12d95a602038cda'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:12.806007 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f282f902-7830-4a09-8453-bc41b37c86d1 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:12.836724 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1029/2091] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:31:12 2022] DELETE /identity/v3/users/a03657e872e94266b12d95a602038cda => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:12.846605 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c703317f-ca2d-438e-9aaa-c6c76c4700de admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:12.851524 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c703317f-ca2d-438e-9aaa-c6c76c4700de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:12.855161 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c98678-1ec7-44f3-a02a-e4f18bff5f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:12.856778 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c98678-1ec7-44f3-a02a-e4f18bff5f41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:12.857271 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c98678-1ec7-44f3-a02a-e4f18bff5f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:12.857798 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c98678-1ec7-44f3-a02a-e4f18bff5f41 admin admin] PATH_INFO: `/v3/projects/1e1fc099cdda42c8aa5a961633e4f66a/users/df71a61377c04027b40b8f3d78b47591/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:12.862158 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c98678-1ec7-44f3-a02a-e4f18bff5f41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e1fc099cdda42c8aa5a961633e4f66a, user_id=df71a61377c04027b40b8f3d78b47591, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:12.867381 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c98678-1ec7-44f3-a02a-e4f18bff5f41 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:12.892312 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1063/2092] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:31:12 2022] PUT /identity/v3/projects/1e1fc099cdda42c8aa5a961633e4f66a/users/df71a61377c04027b40b8f3d78b47591/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:12.905336 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6371dfc-f702-4c12-b619-6498b3ce65f4 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:12.905744 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6371dfc-f702-4c12-b619-6498b3ce65f4 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:12.906044 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6371dfc-f702-4c12-b619-6498b3ce65f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:12.945860 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-f6371dfc-f702-4c12-b619-6498b3ce65f4 None None] MFA Rules not processed for user `df71a61377c04027b40b8f3d78b47591`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:12.947965 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f6371dfc-f702-4c12-b619-6498b3ce65f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:12.965801 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:13.045498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c703317f-ca2d-438e-9aaa-c6c76c4700de admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:13.070106 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1064/2093] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:12 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:31:13.086317 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6371dfc-f702-4c12-b619-6498b3ce65f4 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:13.090968 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f6371dfc-f702-4c12-b619-6498b3ce65f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:13.210061 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c493ab11-61d2-4947-9248-9cc09174211e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef1d49b36cdc4665b82487be97e9dcd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:13.211268 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493ab11-61d2-4947-9248-9cc09174211e service placement] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:13.212029 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493ab11-61d2-4947-9248-9cc09174211e service placement] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:13.212501 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493ab11-61d2-4947-9248-9cc09174211e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:13.217411 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c493ab11-61d2-4947-9248-9cc09174211e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:13.240625 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c493ab11-61d2-4947-9248-9cc09174211e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:13.243797 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c493ab11-61d2-4947-9248-9cc09174211e service placement] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:13.263942 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1030/2094] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:31:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:31:13.310906 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6371dfc-f702-4c12-b619-6498b3ce65f4 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:13.322053 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f02035-42fb-44a2-b96d-4c15f1cba994 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:13.323527 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f02035-42fb-44a2-b96d-4c15f1cba994 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:13.324102 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f02035-42fb-44a2-b96d-4c15f1cba994 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:13.324603 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f02035-42fb-44a2-b96d-4c15f1cba994 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:13.348103 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f02035-42fb-44a2-b96d-4c15f1cba994 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:13.351639 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f02035-42fb-44a2-b96d-4c15f1cba994 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:13.377956 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1065/2095] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:31:13 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:31:13.538370 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d140a49-703b-422a-af21-52639c4c7a21 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:13.539309 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d140a49-703b-422a-af21-52639c4c7a21 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:13.539913 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d140a49-703b-422a-af21-52639c4c7a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:13.557378 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f02035-42fb-44a2-b96d-4c15f1cba994 service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:13.560963 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-71f02035-42fb-44a2-b96d-4c15f1cba994 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:13.653843 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-9d140a49-703b-422a-af21-52639c4c7a21 None None] MFA Rules not processed for user `ec179985fc3b41d188ca69d9e03263d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:13.655865 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9d140a49-703b-422a-af21-52639c4c7a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:13.862509 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f02035-42fb-44a2-b96d-4c15f1cba994 service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:13.884751 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1363a546-0907-4882-9e59-61be1f618bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:13.892073 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363a546-0907-4882-9e59-61be1f618bbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:13.892629 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363a546-0907-4882-9e59-61be1f618bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:13.893261 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363a546-0907-4882-9e59-61be1f618bbd admin admin] PATH_INFO: `/v3/projects/ad7bfb54384a4d648245aaf12c05ade7` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:13.897751 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1363a546-0907-4882-9e59-61be1f618bbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad7bfb54384a4d648245aaf12c05ade7)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:13.902630 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1363a546-0907-4882-9e59-61be1f618bbd admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:13.980150 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1031/2096] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 444 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:31:14.012136 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d140a49-703b-422a-af21-52639c4c7a21 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:14.015310 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9d140a49-703b-422a-af21-52639c4c7a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:14.032503 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:14.045333 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d140a49-703b-422a-af21-52639c4c7a21 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:14.055692 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea83878b-1498-4193-b37f-e51d78e4429c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:14.056749 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea83878b-1498-4193-b37f-e51d78e4429c service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:14.057124 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea83878b-1498-4193-b37f-e51d78e4429c service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:14.057486 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea83878b-1498-4193-b37f-e51d78e4429c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:14.075811 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea83878b-1498-4193-b37f-e51d78e4429c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:14.078313 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea83878b-1498-4193-b37f-e51d78e4429c service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:14.108582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1032/2097] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:31:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:31:14.143866 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1363a546-0907-4882-9e59-61be1f618bbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad7bfb54384a4d648245aaf12c05ade7', 'request_id': 'req-1363a546-0907-4882-9e59-61be1f618bbd'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:14.149788 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:14.167168 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1363a546-0907-4882-9e59-61be1f618bbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad7bfb54384a4d648245aaf12c05ade7', 'request_id': 'req-1363a546-0907-4882-9e59-61be1f618bbd'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:14.188909 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1363a546-0907-4882-9e59-61be1f618bbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad7bfb54384a4d648245aaf12c05ade7', 'request_id': 'req-1363a546-0907-4882-9e59-61be1f618bbd'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:14.237041 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1363a546-0907-4882-9e59-61be1f618bbd admin admin] The token cache is being invalidate because project ad7bfb54384a4d648245aaf12c05ade7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:31:14.237581 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1363a546-0907-4882-9e59-61be1f618bbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:14.240463 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1363a546-0907-4882-9e59-61be1f618bbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:14.244337 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1066/2098] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:31:13 2022] DELETE /identity/v3/projects/ad7bfb54384a4d648245aaf12c05ade7 => generated 0 bytes in 688 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:31:14.252023 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb1e57-5c12-4ba6-898e-acba91255aa0 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:14.252544 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb1e57-5c12-4ba6-898e-acba91255aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:14.252982 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb1e57-5c12-4ba6-898e-acba91255aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:14.297020 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-cecb1e57-5c12-4ba6-898e-acba91255aa0 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:14.299255 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-cecb1e57-5c12-4ba6-898e-acba91255aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:14.480292 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1033/2099] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:31:14.493342 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38720bb6-4a57-4e26-9de7-47176fb8c24a None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:14.494427 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38720bb6-4a57-4e26-9de7-47176fb8c24a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:14.494629 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38720bb6-4a57-4e26-9de7-47176fb8c24a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:14.541445 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-38720bb6-4a57-4e26-9de7-47176fb8c24a None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:14.544038 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-38720bb6-4a57-4e26-9de7-47176fb8c24a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:14.565791 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1067/2100] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:31:14.567671 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecb1e57-5c12-4ba6-898e-acba91255aa0 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:14.595365 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecb1e57-5c12-4ba6-898e-acba91255aa0 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:14.610138 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb985a3-ffc2-461b-bbbd-3da81afc5a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:14.613171 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb985a3-ffc2-461b-bbbd-3da81afc5a18 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:14.614046 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb985a3-ffc2-461b-bbbd-3da81afc5a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:14.614524 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb985a3-ffc2-461b-bbbd-3da81afc5a18 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:14.616238 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb985a3-ffc2-461b-bbbd-3da81afc5a18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:31:14.616709 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb985a3-ffc2-461b-bbbd-3da81afc5a18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:14.620054 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb985a3-ffc2-461b-bbbd-3da81afc5a18 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:14.659111 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1034/2101] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:31:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:31:14.665339 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38720bb6-4a57-4e26-9de7-47176fb8c24a None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:14.681301 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38720bb6-4a57-4e26-9de7-47176fb8c24a None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:14.695519 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d170de2d-7281-400c-825e-d94f08285476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:14.696626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170de2d-7281-400c-825e-d94f08285476 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:14.697103 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170de2d-7281-400c-825e-d94f08285476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:14.697581 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170de2d-7281-400c-825e-d94f08285476 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:14.699144 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d170de2d-7281-400c-825e-d94f08285476 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:14.706562 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d170de2d-7281-400c-825e-d94f08285476 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:14.834871 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1068/2102] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:31:14 2022] POST /identity/v3/projects => generated 379 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:31:14.839798 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb985a3-ffc2-461b-bbbd-3da81afc5a18 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:14.865135 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb985a3-ffc2-461b-bbbd-3da81afc5a18 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:14.964960 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d170de2d-7281-400c-825e-d94f08285476 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:14.975540 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d170de2d-7281-400c-825e-d94f08285476 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:15.076133 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c3a619-6e53-4df2-bfd7-969a9969e892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:15.077439 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c3a619-6e53-4df2-bfd7-969a9969e892 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:15.078070 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c3a619-6e53-4df2-bfd7-969a9969e892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:15.078568 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c3a619-6e53-4df2-bfd7-969a9969e892 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:15.080299 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c3a619-6e53-4df2-bfd7-969a9969e892 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:15.085231 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c3a619-6e53-4df2-bfd7-969a9969e892 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:15.132752 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1035/2103] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:31:14 2022] POST /identity/v3/users => generated 290 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:31:15.142676 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c3a619-6e53-4df2-bfd7-969a9969e892 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:15.164789 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c3a619-6e53-4df2-bfd7-969a9969e892 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:15.165196 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d170de2d-7281-400c-825e-d94f08285476 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:15.175814 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b853b832-8efd-48de-86b8-09c3ee94bfb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:15.176792 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853b832-8efd-48de-86b8-09c3ee94bfb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:15.177355 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853b832-8efd-48de-86b8-09c3ee94bfb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:15.177585 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a421a39-31e6-435a-81f2-3d6dbed2dcbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:15.178226 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853b832-8efd-48de-86b8-09c3ee94bfb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:15.179676 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b853b832-8efd-48de-86b8-09c3ee94bfb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:15.182234 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a421a39-31e6-435a-81f2-3d6dbed2dcbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:15.182494 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b853b832-8efd-48de-86b8-09c3ee94bfb5 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:15.183008 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a421a39-31e6-435a-81f2-3d6dbed2dcbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:15.184482 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a421a39-31e6-435a-81f2-3d6dbed2dcbd admin admin] PATH_INFO: `/v3/users/3c8ea2ff51554db7b5ba525b5818ba5c` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:15.189017 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a421a39-31e6-435a-81f2-3d6dbed2dcbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c8ea2ff51554db7b5ba525b5818ba5c)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:15.193416 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a421a39-31e6-435a-81f2-3d6dbed2dcbd admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:15.210110 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1036/2104] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:31:15 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:31:15.212729 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b853b832-8efd-48de-86b8-09c3ee94bfb5 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:15.236205 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b853b832-8efd-48de-86b8-09c3ee94bfb5 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:15.244531 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f143ca33-d742-438f-a000-c1ee47f6d186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:15.245340 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143ca33-d742-438f-a000-c1ee47f6d186 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:15.245662 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143ca33-d742-438f-a000-c1ee47f6d186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:15.245950 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143ca33-d742-438f-a000-c1ee47f6d186 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:15.246727 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f143ca33-d742-438f-a000-c1ee47f6d186 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:15.250412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f143ca33-d742-438f-a000-c1ee47f6d186 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:15.274734 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1037/2105] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:31:15 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:31:15.289066 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f143ca33-d742-438f-a000-c1ee47f6d186 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:15.306221 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f143ca33-d742-438f-a000-c1ee47f6d186 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:15.315671 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24e7be0-c573-4342-a4b9-e87e466890f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:15.318071 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e7be0-c573-4342-a4b9-e87e466890f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:15.318770 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e7be0-c573-4342-a4b9-e87e466890f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:15.319504 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e7be0-c573-4342-a4b9-e87e466890f0 admin admin] PATH_INFO: `/v3/projects/f8127fd1ce2e4b26af08198e49bb6493/users/9350082cd71c4f61b4aec194bc758825/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:15.326306 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-4a421a39-31e6-435a-81f2-3d6dbed2dcbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c8ea2ff51554db7b5ba525b5818ba5c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:15.345396 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24e7be0-c573-4342-a4b9-e87e466890f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8127fd1ce2e4b26af08198e49bb6493, user_id=9350082cd71c4f61b4aec194bc758825, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:15.350333 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24e7be0-c573-4342-a4b9-e87e466890f0 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:15.364193 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-4a421a39-31e6-435a-81f2-3d6dbed2dcbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c8ea2ff51554db7b5ba525b5818ba5c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:15.366395 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-4a421a39-31e6-435a-81f2-3d6dbed2dcbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c8ea2ff51554db7b5ba525b5818ba5c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:15.367770 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-4a421a39-31e6-435a-81f2-3d6dbed2dcbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c8ea2ff51554db7b5ba525b5818ba5c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:15.391546 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-4a421a39-31e6-435a-81f2-3d6dbed2dcbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c8ea2ff51554db7b5ba525b5818ba5c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:15.403303 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1038/2106] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:31:15 2022] PUT /identity/v3/projects/f8127fd1ce2e4b26af08198e49bb6493/users/9350082cd71c4f61b4aec194bc758825/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:15.415573 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee65e5-fe03-4291-ace4-32115afd35e6 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:15.415935 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee65e5-fe03-4291-ace4-32115afd35e6 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:15.416347 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee65e5-fe03-4291-ace4-32115afd35e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:15.459906 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1069/2107] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:31:14 2022] DELETE /identity/v3/users/3c8ea2ff51554db7b5ba525b5818ba5c => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:15.470503 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d83e88-25e2-40d7-b5ec-f6bafc7fd60c None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:15.470503 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d83e88-25e2-40d7-b5ec-f6bafc7fd60c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:15.470503 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d83e88-25e2-40d7-b5ec-f6bafc7fd60c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:15.492840 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d6ee65e5-fe03-4291-ace4-32115afd35e6 None None] MFA Rules not processed for user `9350082cd71c4f61b4aec194bc758825`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:15.498955 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d6ee65e5-fe03-4291-ace4-32115afd35e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:15.518415 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-18d83e88-25e2-40d7-b5ec-f6bafc7fd60c None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:15.522630 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-18d83e88-25e2-40d7-b5ec-f6bafc7fd60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:15.535157 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:15.584780 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:15.658359 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1039/2108] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:31:15.664857 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee65e5-fe03-4291-ace4-32115afd35e6 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:15.667575 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d6ee65e5-fe03-4291-ace4-32115afd35e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:15.733711 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1070/2109] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:31:15.773155 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d83e88-25e2-40d7-b5ec-f6bafc7fd60c None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:15.776881 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-18d83e88-25e2-40d7-b5ec-f6bafc7fd60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:15.915038 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee65e5-fe03-4291-ace4-32115afd35e6 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:15.935587 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c6856-539f-4999-bbc2-183b22592e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:15.947502 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c6856-539f-4999-bbc2-183b22592e68 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:15.947938 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c6856-539f-4999-bbc2-183b22592e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:15.953399 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c6856-539f-4999-bbc2-183b22592e68 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:15.954743 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97c6856-539f-4999-bbc2-183b22592e68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:15.977095 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97c6856-539f-4999-bbc2-183b22592e68 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:15.982109 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d83e88-25e2-40d7-b5ec-f6bafc7fd60c None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:15.990672 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66838fef-d204-48d2-b97b-ebd2844388e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:15.991426 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66838fef-d204-48d2-b97b-ebd2844388e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:15.991725 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66838fef-d204-48d2-b97b-ebd2844388e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:15.992115 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66838fef-d204-48d2-b97b-ebd2844388e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:16.020129 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66838fef-d204-48d2-b97b-ebd2844388e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:16.023816 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66838fef-d204-48d2-b97b-ebd2844388e7 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:16.047149 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1071/2110] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:31:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:31:16.049556 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf26c6-9f12-4151-919f-2ee29ee935a9 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:16.050055 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf26c6-9f12-4151-919f-2ee29ee935a9 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:16.051164 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf26c6-9f12-4151-919f-2ee29ee935a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:16.051304 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1040/2111] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:31:15 2022] POST /identity/v3/projects => generated 379 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:31:16.057108 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c6856-539f-4999-bbc2-183b22592e68 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:16.096886 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-1edf26c6-9f12-4151-919f-2ee29ee935a9 None None] MFA Rules not processed for user `e60703da011b4ed6b35f19f9ea4cafe7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:16.098264 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-1edf26c6-9f12-4151-919f-2ee29ee935a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:16.299775 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c6856-539f-4999-bbc2-183b22592e68 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:16.315326 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0afe93-f654-4cfb-b26f-903f93aec03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:16.321035 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0afe93-f654-4cfb-b26f-903f93aec03c admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:16.321035 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0afe93-f654-4cfb-b26f-903f93aec03c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:16.321491 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0afe93-f654-4cfb-b26f-903f93aec03c admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:16.325238 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0afe93-f654-4cfb-b26f-903f93aec03c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:16.326747 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1072/2112] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:31:16.334301 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0afe93-f654-4cfb-b26f-903f93aec03c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:16.334446 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8659327c-2569-45a2-b6bc-6d9bef777725 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:16.334446 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8659327c-2569-45a2-b6bc-6d9bef777725 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:16.334446 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8659327c-2569-45a2-b6bc-6d9bef777725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:16.373033 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-8659327c-2569-45a2-b6bc-6d9bef777725 None None] MFA Rules not processed for user `ec179985fc3b41d188ca69d9e03263d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:16.374647 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8659327c-2569-45a2-b6bc-6d9bef777725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:16.386949 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1041/2113] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:31:16 2022] POST /identity/v3/users => generated 296 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:31:16.388695 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0afe93-f654-4cfb-b26f-903f93aec03c admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:16.391952 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7f0afe93-f654-4cfb-b26f-903f93aec03c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:16.436420 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0afe93-f654-4cfb-b26f-903f93aec03c admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:16.456800 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d4777-df2a-476b-a8c9-4c884d92d165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:16.457784 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d4777-df2a-476b-a8c9-4c884d92d165 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:16.458362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d4777-df2a-476b-a8c9-4c884d92d165 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:16.458870 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d4777-df2a-476b-a8c9-4c884d92d165 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:16.460300 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132d4777-df2a-476b-a8c9-4c884d92d165 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:16.466216 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132d4777-df2a-476b-a8c9-4c884d92d165 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:16.520013 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1042/2114] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:31:16 2022] POST /identity/v3/projects => generated 373 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:31:16.521514 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d4777-df2a-476b-a8c9-4c884d92d165 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:16.526230 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1073/2115] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:31:16.529362 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8659327c-2569-45a2-b6bc-6d9bef777725 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:16.537493 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:16.554762 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8659327c-2569-45a2-b6bc-6d9bef777725 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:16.566184 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:16.702361 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d4777-df2a-476b-a8c9-4c884d92d165 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:16.706778 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44846322-cec7-4f80-b32d-706cc47e0998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:16.707501 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44846322-cec7-4f80-b32d-706cc47e0998 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:16.707774 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44846322-cec7-4f80-b32d-706cc47e0998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:16.708052 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44846322-cec7-4f80-b32d-706cc47e0998 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:16.709013 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44846322-cec7-4f80-b32d-706cc47e0998 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:16.711286 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44846322-cec7-4f80-b32d-706cc47e0998 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:16.712625 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3996e119-e985-4deb-8b82-9f3d511113c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:16.715167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996e119-e985-4deb-8b82-9f3d511113c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:16.715167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996e119-e985-4deb-8b82-9f3d511113c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:16.715167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996e119-e985-4deb-8b82-9f3d511113c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:16.717082 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3996e119-e985-4deb-8b82-9f3d511113c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:16.720648 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3996e119-e985-4deb-8b82-9f3d511113c4 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:16.742457 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1074/2116] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:31:16 2022] POST /identity/v3/users => generated 293 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:31:16.750253 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44846322-cec7-4f80-b32d-706cc47e0998 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:16.753816 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1043/2117] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:31:16.755434 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3996e119-e985-4deb-8b82-9f3d511113c4 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:16.774790 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44846322-cec7-4f80-b32d-706cc47e0998 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:16.775435 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3996e119-e985-4deb-8b82-9f3d511113c4 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:16.785307 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ecd104-2874-442e-b37b-656c40676a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:16.786172 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ecd104-2874-442e-b37b-656c40676a70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:16.786732 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ecd104-2874-442e-b37b-656c40676a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:16.787256 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ecd104-2874-442e-b37b-656c40676a70 admin admin] PATH_INFO: `/v3/projects/2f82e21ec6d5411abbbf04bbf1c4a206/users/538e5f8290914151b5d6dfec8c62d27b/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:16.788487 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f598a4-f019-478c-83b5-a5f737b46538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:16.789564 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f598a4-f019-478c-83b5-a5f737b46538 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:16.790154 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f598a4-f019-478c-83b5-a5f737b46538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:16.790543 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f598a4-f019-478c-83b5-a5f737b46538 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:16.791831 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f598a4-f019-478c-83b5-a5f737b46538 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:16.795763 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f598a4-f019-478c-83b5-a5f737b46538 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:16.820001 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1075/2118] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:31:16.824692 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ecd104-2874-442e-b37b-656c40676a70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f82e21ec6d5411abbbf04bbf1c4a206, user_id=538e5f8290914151b5d6dfec8c62d27b, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:16.825347 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f598a4-f019-478c-83b5-a5f737b46538 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:16.830925 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ecd104-2874-442e-b37b-656c40676a70 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:16.844143 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f598a4-f019-478c-83b5-a5f737b46538 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:16.853366 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b53573-37d9-4db6-951d-cc92600d0292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:16.854413 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b53573-37d9-4db6-951d-cc92600d0292 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:16.855511 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b53573-37d9-4db6-951d-cc92600d0292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:16.855895 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b53573-37d9-4db6-951d-cc92600d0292 admin admin] PATH_INFO: `/v3/projects/ae50dbc716e04b0daee1fd5c697dcea7/users/e7e129cf70ab46418e83e7e7488673bb/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:16.870696 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1044/2119] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:31:16 2022] PUT /identity/v3/projects/2f82e21ec6d5411abbbf04bbf1c4a206/users/538e5f8290914151b5d6dfec8c62d27b/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:16.872585 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ecd104-2874-442e-b37b-656c40676a70 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:16.876751 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d0ecd104-2874-442e-b37b-656c40676a70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:16.886008 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b53573-37d9-4db6-951d-cc92600d0292 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae50dbc716e04b0daee1fd5c697dcea7, user_id=e7e129cf70ab46418e83e7e7488673bb, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:16.898470 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b53573-37d9-4db6-951d-cc92600d0292 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:16.956695 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1076/2120] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:31:16 2022] PUT /identity/v3/projects/ae50dbc716e04b0daee1fd5c697dcea7/users/e7e129cf70ab46418e83e7e7488673bb/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:16.958096 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b53573-37d9-4db6-951d-cc92600d0292 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:17.023255 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ecd104-2874-442e-b37b-656c40676a70 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:17.081058 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b53573-37d9-4db6-951d-cc92600d0292 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:17.089133 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d608e36d-75f4-4d3b-a9f8-5f3b75c81cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:17.090336 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608e36d-75f4-4d3b-a9f8-5f3b75c81cbf admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:17.090860 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608e36d-75f4-4d3b-a9f8-5f3b75c81cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:17.092178 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608e36d-75f4-4d3b-a9f8-5f3b75c81cbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:17.093693 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d608e36d-75f4-4d3b-a9f8-5f3b75c81cbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:17.099820 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d608e36d-75f4-4d3b-a9f8-5f3b75c81cbf admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:17.120306 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1077/2121] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:31:17.120838 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d608e36d-75f4-4d3b-a9f8-5f3b75c81cbf admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:17.135699 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d608e36d-75f4-4d3b-a9f8-5f3b75c81cbf admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:17.144082 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94214107-7433-4cb8-9d08-968410c5f381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:17.145407 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94214107-7433-4cb8-9d08-968410c5f381 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:17.145979 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94214107-7433-4cb8-9d08-968410c5f381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:17.147080 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94214107-7433-4cb8-9d08-968410c5f381 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:17.147794 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94214107-7433-4cb8-9d08-968410c5f381 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:17.151396 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94214107-7433-4cb8-9d08-968410c5f381 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:17.160569 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d155d3c9-a1e5-42f2-a438-176e056570fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:17.161834 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155d3c9-a1e5-42f2-a438-176e056570fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:17.162317 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155d3c9-a1e5-42f2-a438-176e056570fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:17.162883 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155d3c9-a1e5-42f2-a438-176e056570fb admin admin] PATH_INFO: `/v3/projects/1a2bc96612ac4302b39f8ffe341551cb` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:17.164897 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d155d3c9-a1e5-42f2-a438-176e056570fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a2bc96612ac4302b39f8ffe341551cb)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:17.168978 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d155d3c9-a1e5-42f2-a438-176e056570fb admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:17.176400 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1078/2122] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:31:17 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:31:17.176400 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94214107-7433-4cb8-9d08-968410c5f381 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:17.194767 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94214107-7433-4cb8-9d08-968410c5f381 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:17.203510 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0779aa-cc49-4eb9-931f-1db639bb30a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:17.205121 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0779aa-cc49-4eb9-931f-1db639bb30a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:17.205578 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0779aa-cc49-4eb9-931f-1db639bb30a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:17.206100 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0779aa-cc49-4eb9-931f-1db639bb30a6 admin admin] PATH_INFO: `/v3/projects/2f82e21ec6d5411abbbf04bbf1c4a206/users/538e5f8290914151b5d6dfec8c62d27b/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:17.210098 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0779aa-cc49-4eb9-931f-1db639bb30a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f82e21ec6d5411abbbf04bbf1c4a206, user_id=538e5f8290914151b5d6dfec8c62d27b, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:17.216083 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0779aa-cc49-4eb9-931f-1db639bb30a6 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:17.245497 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:17.266215 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1079/2123] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:31:17 2022] PUT /identity/v3/projects/2f82e21ec6d5411abbbf04bbf1c4a206/users/538e5f8290914151b5d6dfec8c62d27b/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:17.278851 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0779aa-cc49-4eb9-931f-1db639bb30a6 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:17.293592 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d155d3c9-a1e5-42f2-a438-176e056570fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a2bc96612ac4302b39f8ffe341551cb', 'request_id': 'req-d155d3c9-a1e5-42f2-a438-176e056570fb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:17.320965 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d155d3c9-a1e5-42f2-a438-176e056570fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a2bc96612ac4302b39f8ffe341551cb', 'request_id': 'req-d155d3c9-a1e5-42f2-a438-176e056570fb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:17.340241 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d155d3c9-a1e5-42f2-a438-176e056570fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a2bc96612ac4302b39f8ffe341551cb', 'request_id': 'req-d155d3c9-a1e5-42f2-a438-176e056570fb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:17.361551 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:17.371407 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d155d3c9-a1e5-42f2-a438-176e056570fb admin admin] The token cache is being invalidate because project 1a2bc96612ac4302b39f8ffe341551cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:31:17.371923 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d155d3c9-a1e5-42f2-a438-176e056570fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:17.373458 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d155d3c9-a1e5-42f2-a438-176e056570fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:17.376547 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1045/2124] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:31:16 2022] DELETE /identity/v3/projects/1a2bc96612ac4302b39f8ffe341551cb => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:17.382173 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96209c9-d875-4db1-bc55-233acb199371 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:17.383648 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96209c9-d875-4db1-bc55-233acb199371 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:17.385268 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96209c9-d875-4db1-bc55-233acb199371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:17.442233 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0779aa-cc49-4eb9-931f-1db639bb30a6 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:17.464521 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-c96209c9-d875-4db1-bc55-233acb199371 None None] MFA Rules not processed for user `538e5f8290914151b5d6dfec8c62d27b`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:17.467189 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c96209c9-d875-4db1-bc55-233acb199371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:17.628560 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a637be44-d9e9-43c2-89be-145d3b6e2c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:17.629993 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a637be44-d9e9-43c2-89be-145d3b6e2c66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:17.630413 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a637be44-d9e9-43c2-89be-145d3b6e2c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:17.630843 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a637be44-d9e9-43c2-89be-145d3b6e2c66 admin admin] PATH_INFO: `/v3/projects/ae50dbc716e04b0daee1fd5c697dcea7/users/e7e129cf70ab46418e83e7e7488673bb/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:17.635040 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a637be44-d9e9-43c2-89be-145d3b6e2c66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae50dbc716e04b0daee1fd5c697dcea7, user_id=e7e129cf70ab46418e83e7e7488673bb, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:17.640944 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a637be44-d9e9-43c2-89be-145d3b6e2c66 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:17.670769 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1046/2125] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:17 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:31:17.671912 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96209c9-d875-4db1-bc55-233acb199371 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:17.674234 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c96209c9-d875-4db1-bc55-233acb199371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:17.699675 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1080/2126] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:31:17 2022] PUT /identity/v3/projects/ae50dbc716e04b0daee1fd5c697dcea7/users/e7e129cf70ab46418e83e7e7488673bb/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:17.701219 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a637be44-d9e9-43c2-89be-145d3b6e2c66 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:17.711257 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a637be44-d9e9-43c2-89be-145d3b6e2c66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:17.716155 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96209c9-d875-4db1-bc55-233acb199371 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:17.870416 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb6e9d-4bb0-446e-aa4e-28f5a3b6aeb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:17.874381 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb6e9d-4bb0-446e-aa4e-28f5a3b6aeb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:17.876049 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb6e9d-4bb0-446e-aa4e-28f5a3b6aeb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:17.877371 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb6e9d-4bb0-446e-aa4e-28f5a3b6aeb8 admin admin] PATH_INFO: `/v3/users/df71a61377c04027b40b8f3d78b47591` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:17.884801 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb6e9d-4bb0-446e-aa4e-28f5a3b6aeb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df71a61377c04027b40b8f3d78b47591)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:17.892700 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb6e9d-4bb0-446e-aa4e-28f5a3b6aeb8 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:17.894508 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a637be44-d9e9-43c2-89be-145d3b6e2c66 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:17.906747 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fb73d8-1f67-42da-be02-20336e1cec5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:17.909682 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb73d8-1f67-42da-be02-20336e1cec5c service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:17.910133 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb73d8-1f67-42da-be02-20336e1cec5c service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:17.910980 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb73d8-1f67-42da-be02-20336e1cec5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:18.078086 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:18.113481 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2eeb6e9d-4bb0-446e-aa4e-28f5a3b6aeb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df71a61377c04027b40b8f3d78b47591'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:18.137396 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2eeb6e9d-4bb0-446e-aa4e-28f5a3b6aeb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df71a61377c04027b40b8f3d78b47591'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:18.139811 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2eeb6e9d-4bb0-446e-aa4e-28f5a3b6aeb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df71a61377c04027b40b8f3d78b47591'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:18.143272 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2eeb6e9d-4bb0-446e-aa4e-28f5a3b6aeb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df71a61377c04027b40b8f3d78b47591'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:18.210320 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fb73d8-1f67-42da-be02-20336e1cec5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:18.216522 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2eeb6e9d-4bb0-446e-aa4e-28f5a3b6aeb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df71a61377c04027b40b8f3d78b47591'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:18.221887 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fb73d8-1f67-42da-be02-20336e1cec5c service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:18.248674 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1047/2127] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:31:17 2022] DELETE /identity/v3/users/df71a61377c04027b40b8f3d78b47591 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:18.271756 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1081/2128] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:31:17 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:31:18.455465 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb6e9d-4bb0-446e-aa4e-28f5a3b6aeb8 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:18.456886 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2eeb6e9d-4bb0-446e-aa4e-28f5a3b6aeb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:18.615639 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb6e9d-4bb0-446e-aa4e-28f5a3b6aeb8 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:18.623514 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2334b230-9a53-4f25-ad37-7584b53330c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:18.626318 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334b230-9a53-4f25-ad37-7584b53330c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:18.627499 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334b230-9a53-4f25-ad37-7584b53330c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:18.629837 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334b230-9a53-4f25-ad37-7584b53330c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:18.636126 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2334b230-9a53-4f25-ad37-7584b53330c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:18.905452 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2334b230-9a53-4f25-ad37-7584b53330c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:18.908239 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2334b230-9a53-4f25-ad37-7584b53330c4 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:18.936204 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1048/2129] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:31:18 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:31:19.259503 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fb73d8-1f67-42da-be02-20336e1cec5c service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:19.297680 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fb73d8-1f67-42da-be02-20336e1cec5c service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:19.368980 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:19.469222 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883bc343-3f06-4ce7-8e56-6745715692fb None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:19.470315 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883bc343-3f06-4ce7-8e56-6745715692fb None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:19.470315 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883bc343-3f06-4ce7-8e56-6745715692fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:19.472489 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f44fc47-5927-4337-a89a-f58338f1f2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:19.474813 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44fc47-5927-4337-a89a-f58338f1f2ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:19.475303 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44fc47-5927-4337-a89a-f58338f1f2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:19.476123 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44fc47-5927-4337-a89a-f58338f1f2ce admin admin] PATH_INFO: `/v3/projects/1e1fc099cdda42c8aa5a961633e4f66a` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:19.480769 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f44fc47-5927-4337-a89a-f58338f1f2ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e1fc099cdda42c8aa5a961633e4f66a)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:19.485020 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f44fc47-5927-4337-a89a-f58338f1f2ce admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:19.539981 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-883bc343-3f06-4ce7-8e56-6745715692fb None None] MFA Rules not processed for user `e7e129cf70ab46418e83e7e7488673bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:19.544258 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-883bc343-3f06-4ce7-8e56-6745715692fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:19.619994 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9f44fc47-5927-4337-a89a-f58338f1f2ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e1fc099cdda42c8aa5a961633e4f66a', 'request_id': 'req-9f44fc47-5927-4337-a89a-f58338f1f2ce'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:19.642797 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9f44fc47-5927-4337-a89a-f58338f1f2ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e1fc099cdda42c8aa5a961633e4f66a', 'request_id': 'req-9f44fc47-5927-4337-a89a-f58338f1f2ce'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:19.660687 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9f44fc47-5927-4337-a89a-f58338f1f2ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e1fc099cdda42c8aa5a961633e4f66a', 'request_id': 'req-9f44fc47-5927-4337-a89a-f58338f1f2ce'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:19.688200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9f44fc47-5927-4337-a89a-f58338f1f2ce admin admin] The token cache is being invalidate because project 1e1fc099cdda42c8aa5a961633e4f66a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:31:19.688775 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9f44fc47-5927-4337-a89a-f58338f1f2ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:19.690339 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9f44fc47-5927-4337-a89a-f58338f1f2ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:19.693805 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1082/2130] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:31:19 2022] DELETE /identity/v3/projects/1e1fc099cdda42c8aa5a961633e4f66a => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:19.701907 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f320b-80dd-432f-b0d2-6ba767d75215 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:19.702485 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f320b-80dd-432f-b0d2-6ba767d75215 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:19.702980 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f320b-80dd-432f-b0d2-6ba767d75215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:19.731123 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:19.739893 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-2d4f320b-80dd-432f-b0d2-6ba767d75215 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:19.742604 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1049/2131] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:19 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:31:19.742842 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-2d4f320b-80dd-432f-b0d2-6ba767d75215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:19.762151 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883bc343-3f06-4ce7-8e56-6745715692fb None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:19.765386 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-883bc343-3f06-4ce7-8e56-6745715692fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:19.787622 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883bc343-3f06-4ce7-8e56-6745715692fb None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:19.871843 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1083/2132] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:31:19.885119 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e31b41-af83-4546-829b-cbcb11dcdd64 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:19.885564 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e31b41-af83-4546-829b-cbcb11dcdd64 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:19.886055 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e31b41-af83-4546-829b-cbcb11dcdd64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:19.899659 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a05a7a3-2906-4c55-bbf3-f46c7fc71702 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:19.900632 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05a7a3-2906-4c55-bbf3-f46c7fc71702 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:19.901051 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05a7a3-2906-4c55-bbf3-f46c7fc71702 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:19.901425 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05a7a3-2906-4c55-bbf3-f46c7fc71702 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:19.907025 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2a05a7a3-2906-4c55-bbf3-f46c7fc71702 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:19.921500 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-88e31b41-af83-4546-829b-cbcb11dcdd64 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:19.924365 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-88e31b41-af83-4546-829b-cbcb11dcdd64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:19.939071 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1084/2133] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:31:19.946883 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e31b41-af83-4546-829b-cbcb11dcdd64 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:19.959095 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e31b41-af83-4546-829b-cbcb11dcdd64 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:19.966218 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0def08d7-d9b5-4690-8ef5-e4ce14a40b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:19.967467 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def08d7-d9b5-4690-8ef5-e4ce14a40b66 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:19.968035 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def08d7-d9b5-4690-8ef5-e4ce14a40b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:19.968517 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def08d7-d9b5-4690-8ef5-e4ce14a40b66 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:19.969813 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0def08d7-d9b5-4690-8ef5-e4ce14a40b66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:31:19.970168 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0def08d7-d9b5-4690-8ef5-e4ce14a40b66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:19.973318 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0def08d7-d9b5-4690-8ef5-e4ce14a40b66 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:19.997864 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1085/2134] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:31:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:31:20.004406 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0def08d7-d9b5-4690-8ef5-e4ce14a40b66 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:20.022062 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0def08d7-d9b5-4690-8ef5-e4ce14a40b66 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:20.032999 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2a6e11-3907-4078-b267-1a070d16a684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:20.033908 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a6e11-3907-4078-b267-1a070d16a684 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:20.034388 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a6e11-3907-4078-b267-1a070d16a684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:20.034912 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a6e11-3907-4078-b267-1a070d16a684 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:20.036154 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2a6e11-3907-4078-b267-1a070d16a684 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:20.038972 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a05a7a3-2906-4c55-bbf3-f46c7fc71702 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:20.039670 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2a6e11-3907-4078-b267-1a070d16a684 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:20.041857 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a05a7a3-2906-4c55-bbf3-f46c7fc71702 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:20.062883 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1050/2135] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:31:19 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:31:20.090683 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1086/2136] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:31:20 2022] POST /identity/v3/projects => generated 381 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:31:20.096974 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a05a7a3-2906-4c55-bbf3-f46c7fc71702 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:20.121139 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a05a7a3-2906-4c55-bbf3-f46c7fc71702 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:20.242813 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088d04f6-c0f9-42ef-ab3d-36de255ecd36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:20.243536 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d04f6-c0f9-42ef-ab3d-36de255ecd36 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:20.243827 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d04f6-c0f9-42ef-ab3d-36de255ecd36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:20.244113 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d04f6-c0f9-42ef-ab3d-36de255ecd36 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:20.244837 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088d04f6-c0f9-42ef-ab3d-36de255ecd36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:20.247212 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088d04f6-c0f9-42ef-ab3d-36de255ecd36 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:20.291596 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1051/2137] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:31:20 2022] POST /identity/v3/users => generated 291 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:31:20.293759 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2a6e11-3907-4078-b267-1a070d16a684 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:20.311902 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2a6e11-3907-4078-b267-1a070d16a684 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:20.318349 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b50183-2537-4725-85c6-29784c00fe95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:20.320194 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b50183-2537-4725-85c6-29784c00fe95 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:20.320569 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b50183-2537-4725-85c6-29784c00fe95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:20.320938 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b50183-2537-4725-85c6-29784c00fe95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:20.321843 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b50183-2537-4725-85c6-29784c00fe95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:20.323777 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b50183-2537-4725-85c6-29784c00fe95 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:20.341226 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1087/2138] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:31:20 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:31:20.347522 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088d04f6-c0f9-42ef-ab3d-36de255ecd36 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:20.362677 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088d04f6-c0f9-42ef-ab3d-36de255ecd36 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:20.370625 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bbe74a-7775-4375-a822-4e47038cb841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:20.373487 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bbe74a-7775-4375-a822-4e47038cb841 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:20.374181 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bbe74a-7775-4375-a822-4e47038cb841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:20.374819 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bbe74a-7775-4375-a822-4e47038cb841 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:20.376051 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bbe74a-7775-4375-a822-4e47038cb841 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:20.378197 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bbe74a-7775-4375-a822-4e47038cb841 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:20.406034 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1052/2139] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:31:20 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:31:20.411648 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b50183-2537-4725-85c6-29784c00fe95 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:20.425169 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:20.433347 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b50183-2537-4725-85c6-29784c00fe95 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:20.470384 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec08deb-198e-430b-84a8-c80358b1d1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:20.473129 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec08deb-198e-430b-84a8-c80358b1d1f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:20.476399 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec08deb-198e-430b-84a8-c80358b1d1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:20.477626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec08deb-198e-430b-84a8-c80358b1d1f5 admin admin] PATH_INFO: `/v3/projects/e8a78f5d89b547749ba140bfa657b053/users/f047d3d2e6934b4492d2f272d1e34869/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:20.536252 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec08deb-198e-430b-84a8-c80358b1d1f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8a78f5d89b547749ba140bfa657b053, user_id=f047d3d2e6934b4492d2f272d1e34869, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:20.541922 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec08deb-198e-430b-84a8-c80358b1d1f5 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:20.577551 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1088/2140] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:31:20 2022] PUT /identity/v3/projects/e8a78f5d89b547749ba140bfa657b053/users/f047d3d2e6934b4492d2f272d1e34869/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:20.597759 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09905f34-fb4e-412a-8825-27aa2604fb1d None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:20.598427 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09905f34-fb4e-412a-8825-27aa2604fb1d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:20.598427 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09905f34-fb4e-412a-8825-27aa2604fb1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:20.690120 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-09905f34-fb4e-412a-8825-27aa2604fb1d None None] MFA Rules not processed for user `f047d3d2e6934b4492d2f272d1e34869`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:20.692606 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-09905f34-fb4e-412a-8825-27aa2604fb1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:20.746745 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:20.864364 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1053/2141] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:31:20.872843 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec08deb-198e-430b-84a8-c80358b1d1f5 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:21.013586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec08deb-198e-430b-84a8-c80358b1d1f5 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:21.023139 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7861280e-65a7-45c2-97c3-5fc3d93336fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:21.024301 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861280e-65a7-45c2-97c3-5fc3d93336fb admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:21.024619 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861280e-65a7-45c2-97c3-5fc3d93336fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:21.025101 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861280e-65a7-45c2-97c3-5fc3d93336fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:21.026148 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7861280e-65a7-45c2-97c3-5fc3d93336fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:21.028816 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7861280e-65a7-45c2-97c3-5fc3d93336fb admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:21.152313 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1089/2142] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:31:20 2022] POST /identity/v3/projects => generated 381 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:31:21.157884 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09905f34-fb4e-412a-8825-27aa2604fb1d None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:21.192069 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09905f34-fb4e-412a-8825-27aa2604fb1d None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:21.356845 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e1ab7c-41c7-4d02-89e0-b573ca065cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:21.357796 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e1ab7c-41c7-4d02-89e0-b573ca065cb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:21.358178 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e1ab7c-41c7-4d02-89e0-b573ca065cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:21.358550 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e1ab7c-41c7-4d02-89e0-b573ca065cb6 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:21.359547 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e1ab7c-41c7-4d02-89e0-b573ca065cb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:21.362167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e1ab7c-41c7-4d02-89e0-b573ca065cb6 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:21.392196 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1054/2143] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:31:21 2022] POST /identity/v3/users => generated 297 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:31:21.398490 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7861280e-65a7-45c2-97c3-5fc3d93336fb admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:21.422614 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7861280e-65a7-45c2-97c3-5fc3d93336fb admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:21.432639 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30b8449-780c-43cc-850b-7665f8c090b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:21.435107 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b8449-780c-43cc-850b-7665f8c090b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:21.435107 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b8449-780c-43cc-850b-7665f8c090b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:21.435107 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b8449-780c-43cc-850b-7665f8c090b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:21.436262 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30b8449-780c-43cc-850b-7665f8c090b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:21.440457 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30b8449-780c-43cc-850b-7665f8c090b8 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:21.459208 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1090/2144] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:31:21 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:31:21.465239 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e1ab7c-41c7-4d02-89e0-b573ca065cb6 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:21.482489 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e1ab7c-41c7-4d02-89e0-b573ca065cb6 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:21.491912 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7112a8-2f90-49e7-be7d-c4de5f1b75b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:21.492700 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7112a8-2f90-49e7-be7d-c4de5f1b75b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:21.493245 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7112a8-2f90-49e7-be7d-c4de5f1b75b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:21.493779 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7112a8-2f90-49e7-be7d-c4de5f1b75b7 admin admin] PATH_INFO: `/v3/projects/98b0014c3f4343258a7c16fce79997eb/users/18e4e792995944449dc7c125dcaedf21/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:21.525336 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7112a8-2f90-49e7-be7d-c4de5f1b75b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98b0014c3f4343258a7c16fce79997eb, user_id=18e4e792995944449dc7c125dcaedf21, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:21.529696 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7112a8-2f90-49e7-be7d-c4de5f1b75b7 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:21.553299 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1055/2145] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:31:21 2022] PUT /identity/v3/projects/98b0014c3f4343258a7c16fce79997eb/users/18e4e792995944449dc7c125dcaedf21/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:21.557844 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30b8449-780c-43cc-850b-7665f8c090b8 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:21.579066 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30b8449-780c-43cc-850b-7665f8c090b8 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:21.696467 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca5727c-4bd3-473a-9bca-eb193be05160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:21.697174 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca5727c-4bd3-473a-9bca-eb193be05160 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:21.697448 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca5727c-4bd3-473a-9bca-eb193be05160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:21.697766 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca5727c-4bd3-473a-9bca-eb193be05160 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:21.698522 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca5727c-4bd3-473a-9bca-eb193be05160 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:21.700469 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca5727c-4bd3-473a-9bca-eb193be05160 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:21.719358 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1091/2146] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:31:21 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:31:21.726382 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7112a8-2f90-49e7-be7d-c4de5f1b75b7 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:21.749895 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7112a8-2f90-49e7-be7d-c4de5f1b75b7 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:21.758408 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b1fc1-4da4-41aa-a68e-e8911837ed3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:21.760092 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b1fc1-4da4-41aa-a68e-e8911837ed3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:21.760606 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b1fc1-4da4-41aa-a68e-e8911837ed3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:21.761104 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b1fc1-4da4-41aa-a68e-e8911837ed3b admin admin] PATH_INFO: `/v3/projects/98b0014c3f4343258a7c16fce79997eb/users/18e4e792995944449dc7c125dcaedf21/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:21.765571 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2b1fc1-4da4-41aa-a68e-e8911837ed3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=98b0014c3f4343258a7c16fce79997eb, user_id=18e4e792995944449dc7c125dcaedf21, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:21.772383 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2b1fc1-4da4-41aa-a68e-e8911837ed3b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:21.794928 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1056/2147] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:31:21 2022] PUT /identity/v3/projects/98b0014c3f4343258a7c16fce79997eb/users/18e4e792995944449dc7c125dcaedf21/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:21.813957 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4015e07-7a7f-4b1c-a5ea-ac7eb1da99b4 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:21.814403 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4015e07-7a7f-4b1c-a5ea-ac7eb1da99b4 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:21.814828 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4015e07-7a7f-4b1c-a5ea-ac7eb1da99b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:21.876827 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-a4015e07-7a7f-4b1c-a5ea-ac7eb1da99b4 None None] MFA Rules not processed for user `18e4e792995944449dc7c125dcaedf21`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:21.878423 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a4015e07-7a7f-4b1c-a5ea-ac7eb1da99b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:21.980126 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:22.002774 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b1fc1-4da4-41aa-a68e-e8911837ed3b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:22.034871 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1092/2148] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:21 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:31:22.054184 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4015e07-7a7f-4b1c-a5ea-ac7eb1da99b4 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:22.056676 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a4015e07-7a7f-4b1c-a5ea-ac7eb1da99b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:22.118748 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b1fc1-4da4-41aa-a68e-e8911837ed3b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:22.126677 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7af660-0e7b-4a1d-8f97-9af27a7f19a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:22.127816 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7af660-0e7b-4a1d-8f97-9af27a7f19a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:22.128307 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7af660-0e7b-4a1d-8f97-9af27a7f19a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:22.128798 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7af660-0e7b-4a1d-8f97-9af27a7f19a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:22.193094 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:22.207371 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4015e07-7a7f-4b1c-a5ea-ac7eb1da99b4 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:22.216280 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc3049-3dbd-4929-8226-34425bac185b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:22.217346 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc3049-3dbd-4929-8226-34425bac185b service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:22.217888 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc3049-3dbd-4929-8226-34425bac185b service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:22.218187 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc3049-3dbd-4929-8226-34425bac185b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:22.257881 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc3049-3dbd-4929-8226-34425bac185b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:22.260690 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc3049-3dbd-4929-8226-34425bac185b service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:22.292920 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7af660-0e7b-4a1d-8f97-9af27a7f19a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:22.296413 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7af660-0e7b-4a1d-8f97-9af27a7f19a0 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:22.301947 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1093/2149] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:31:22 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:31:22.319246 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1057/2150] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:31:22 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:31:22.728634 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc3049-3dbd-4929-8226-34425bac185b service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:22.742957 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc3049-3dbd-4929-8226-34425bac185b service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:22.752264 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3654db2-23b4-4f92-be4d-54685fc7df9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:22.752264 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3654db2-23b4-4f92-be4d-54685fc7df9e service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:22.752816 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3654db2-23b4-4f92-be4d-54685fc7df9e service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:22.753329 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3654db2-23b4-4f92-be4d-54685fc7df9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:22.885734 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3654db2-23b4-4f92-be4d-54685fc7df9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:22.887995 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3654db2-23b4-4f92-be4d-54685fc7df9e service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:22.912236 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1094/2151] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:31:22 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:31:23.572608 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7af660-0e7b-4a1d-8f97-9af27a7f19a0 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:23.663384 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7af660-0e7b-4a1d-8f97-9af27a7f19a0 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:23.669147 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d1e8af-83a7-455f-8344-6c8fdabbe4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:23.670619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1e8af-83a7-455f-8344-6c8fdabbe4ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:23.671009 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1e8af-83a7-455f-8344-6c8fdabbe4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:23.671374 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1e8af-83a7-455f-8344-6c8fdabbe4ea admin admin] PATH_INFO: `/v3/users/f047d3d2e6934b4492d2f272d1e34869` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:23.673344 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d1e8af-83a7-455f-8344-6c8fdabbe4ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=f047d3d2e6934b4492d2f272d1e34869)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:23.675533 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d1e8af-83a7-455f-8344-6c8fdabbe4ea admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:23.780959 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-08d1e8af-83a7-455f-8344-6c8fdabbe4ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f047d3d2e6934b4492d2f272d1e34869'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:23.792821 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-08d1e8af-83a7-455f-8344-6c8fdabbe4ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f047d3d2e6934b4492d2f272d1e34869'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:23.796414 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-08d1e8af-83a7-455f-8344-6c8fdabbe4ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f047d3d2e6934b4492d2f272d1e34869'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:23.797219 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-08d1e8af-83a7-455f-8344-6c8fdabbe4ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f047d3d2e6934b4492d2f272d1e34869'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:23.824884 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-08d1e8af-83a7-455f-8344-6c8fdabbe4ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f047d3d2e6934b4492d2f272d1e34869'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:23.848823 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1058/2152] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:31:23 2022] DELETE /identity/v3/users/f047d3d2e6934b4492d2f272d1e34869 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:23.854081 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f77ea-d875-455b-8f32-8fc69441ac6d None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:23.855871 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f77ea-d875-455b-8f32-8fc69441ac6d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:23.856512 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f77ea-d875-455b-8f32-8fc69441ac6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:23.888605 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-134f77ea-d875-455b-8f32-8fc69441ac6d None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:23.890583 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-134f77ea-d875-455b-8f32-8fc69441ac6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:23.999936 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1095/2153] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:31:24.022504 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d1e8af-83a7-455f-8344-6c8fdabbe4ea admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:24.027644 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-08d1e8af-83a7-455f-8344-6c8fdabbe4ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:24.135772 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:24.201282 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d1e8af-83a7-455f-8344-6c8fdabbe4ea admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:24.208773 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d53d26-0690-49d9-a516-3892c1ed02c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:24.210044 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d53d26-0690-49d9-a516-3892c1ed02c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:24.210848 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d53d26-0690-49d9-a516-3892c1ed02c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:24.211453 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d53d26-0690-49d9-a516-3892c1ed02c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:24.245918 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d53d26-0690-49d9-a516-3892c1ed02c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:24.249335 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d53d26-0690-49d9-a516-3892c1ed02c4 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:24.274025 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1059/2154] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:31:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:31:24.727949 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134f77ea-d875-455b-8f32-8fc69441ac6d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:24.730721 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-134f77ea-d875-455b-8f32-8fc69441ac6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:24.754802 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134f77ea-d875-455b-8f32-8fc69441ac6d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:24.763305 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5dd401-23e7-4267-bcaf-591b2d2b0bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:24.764847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5dd401-23e7-4267-bcaf-591b2d2b0bb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:24.765361 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5dd401-23e7-4267-bcaf-591b2d2b0bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:24.765894 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5dd401-23e7-4267-bcaf-591b2d2b0bb9 admin admin] PATH_INFO: `/v3/projects/e8a78f5d89b547749ba140bfa657b053` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:24.768207 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5dd401-23e7-4267-bcaf-591b2d2b0bb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8a78f5d89b547749ba140bfa657b053)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:24.771924 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5dd401-23e7-4267-bcaf-591b2d2b0bb9 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:24.795291 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:24.852061 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-da5dd401-23e7-4267-bcaf-591b2d2b0bb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8a78f5d89b547749ba140bfa657b053', 'request_id': 'req-da5dd401-23e7-4267-bcaf-591b2d2b0bb9'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:24.860529 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-da5dd401-23e7-4267-bcaf-591b2d2b0bb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8a78f5d89b547749ba140bfa657b053', 'request_id': 'req-da5dd401-23e7-4267-bcaf-591b2d2b0bb9'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:24.872602 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-da5dd401-23e7-4267-bcaf-591b2d2b0bb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8a78f5d89b547749ba140bfa657b053', 'request_id': 'req-da5dd401-23e7-4267-bcaf-591b2d2b0bb9'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:24.890581 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-da5dd401-23e7-4267-bcaf-591b2d2b0bb9 admin admin] The token cache is being invalidate because project e8a78f5d89b547749ba140bfa657b053 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:31:24.891059 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-da5dd401-23e7-4267-bcaf-591b2d2b0bb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:24.892144 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-da5dd401-23e7-4267-bcaf-591b2d2b0bb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:24.894098 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1096/2155] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:31:24 2022] DELETE /identity/v3/projects/e8a78f5d89b547749ba140bfa657b053 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:24.899067 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d53d26-0690-49d9-a516-3892c1ed02c4 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:24.912886 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d53d26-0690-49d9-a516-3892c1ed02c4 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:25.017243 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9345eedc-21bf-4dfe-bcf6-3fb9115ede05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:25.018375 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345eedc-21bf-4dfe-bcf6-3fb9115ede05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:25.018841 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345eedc-21bf-4dfe-bcf6-3fb9115ede05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:25.019216 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345eedc-21bf-4dfe-bcf6-3fb9115ede05 admin admin] PATH_INFO: `/v3/users/18e4e792995944449dc7c125dcaedf21` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:25.020811 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9345eedc-21bf-4dfe-bcf6-3fb9115ede05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18e4e792995944449dc7c125dcaedf21)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:25.022989 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9345eedc-21bf-4dfe-bcf6-3fb9115ede05 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:25.101914 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9345eedc-21bf-4dfe-bcf6-3fb9115ede05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18e4e792995944449dc7c125dcaedf21'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:25.111789 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9345eedc-21bf-4dfe-bcf6-3fb9115ede05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18e4e792995944449dc7c125dcaedf21'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:25.112647 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9345eedc-21bf-4dfe-bcf6-3fb9115ede05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18e4e792995944449dc7c125dcaedf21'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:25.113346 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9345eedc-21bf-4dfe-bcf6-3fb9115ede05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18e4e792995944449dc7c125dcaedf21'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:25.129721 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9345eedc-21bf-4dfe-bcf6-3fb9115ede05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18e4e792995944449dc7c125dcaedf21'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:25.146897 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1060/2156] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:31:24 2022] DELETE /identity/v3/users/18e4e792995944449dc7c125dcaedf21 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:31:25.630063 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5dd401-23e7-4267-bcaf-591b2d2b0bb9 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:25.632087 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-da5dd401-23e7-4267-bcaf-591b2d2b0bb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:25.650778 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5dd401-23e7-4267-bcaf-591b2d2b0bb9 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:25.724515 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54900f46-8258-42c1-aad0-16635e8b7b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:25.725448 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54900f46-8258-42c1-aad0-16635e8b7b73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:25.725797 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54900f46-8258-42c1-aad0-16635e8b7b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:25.726131 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54900f46-8258-42c1-aad0-16635e8b7b73 admin admin] PATH_INFO: `/v3/projects/98b0014c3f4343258a7c16fce79997eb` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:25.727514 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54900f46-8258-42c1-aad0-16635e8b7b73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98b0014c3f4343258a7c16fce79997eb)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:25.729900 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54900f46-8258-42c1-aad0-16635e8b7b73 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:25.797033 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-54900f46-8258-42c1-aad0-16635e8b7b73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98b0014c3f4343258a7c16fce79997eb', 'request_id': 'req-54900f46-8258-42c1-aad0-16635e8b7b73'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:25.807391 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-54900f46-8258-42c1-aad0-16635e8b7b73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98b0014c3f4343258a7c16fce79997eb', 'request_id': 'req-54900f46-8258-42c1-aad0-16635e8b7b73'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:25.822892 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-54900f46-8258-42c1-aad0-16635e8b7b73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98b0014c3f4343258a7c16fce79997eb', 'request_id': 'req-54900f46-8258-42c1-aad0-16635e8b7b73'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:25.843600 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-54900f46-8258-42c1-aad0-16635e8b7b73 admin admin] The token cache is being invalidate because project 98b0014c3f4343258a7c16fce79997eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:31:25.843924 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-54900f46-8258-42c1-aad0-16635e8b7b73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:25.844678 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-54900f46-8258-42c1-aad0-16635e8b7b73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:25.847533 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1097/2157] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:31:25 2022] DELETE /identity/v3/projects/98b0014c3f4343258a7c16fce79997eb => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:25.854098 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce02f78-5506-45bc-a388-5f3a7f804783 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:25.855081 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce02f78-5506-45bc-a388-5f3a7f804783 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:25.855561 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce02f78-5506-45bc-a388-5f3a7f804783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:25.887833 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-fce02f78-5506-45bc-a388-5f3a7f804783 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:25.889382 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fce02f78-5506-45bc-a388-5f3a7f804783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:25.963000 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1061/2158] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:31:25.976244 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c8ea3-89f4-4858-8ce1-15962434e408 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:25.977113 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c8ea3-89f4-4858-8ce1-15962434e408 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:25.977113 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c8ea3-89f4-4858-8ce1-15962434e408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:25.998329 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-0d5c8ea3-89f4-4858-8ce1-15962434e408 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:26.000156 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-0d5c8ea3-89f4-4858-8ce1-15962434e408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:26.010053 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1098/2159] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:31:26.014766 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce02f78-5506-45bc-a388-5f3a7f804783 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:26.023174 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:26.031955 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce02f78-5506-45bc-a388-5f3a7f804783 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:26.040851 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9decd8a1-c24c-4354-9cb1-c28677b36146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:26.041711 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9decd8a1-c24c-4354-9cb1-c28677b36146 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:26.042433 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9decd8a1-c24c-4354-9cb1-c28677b36146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:26.042756 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9decd8a1-c24c-4354-9cb1-c28677b36146 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:26.043555 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9decd8a1-c24c-4354-9cb1-c28677b36146 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:31:26.043988 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9decd8a1-c24c-4354-9cb1-c28677b36146 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:26.046208 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9decd8a1-c24c-4354-9cb1-c28677b36146 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:26.063029 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1062/2160] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:31:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:31:26.068953 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5c8ea3-89f4-4858-8ce1-15962434e408 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:26.080606 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5c8ea3-89f4-4858-8ce1-15962434e408 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:26.088322 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96042e31-6b5c-49b2-9021-121d1a5455e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:26.089008 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96042e31-6b5c-49b2-9021-121d1a5455e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:26.089265 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96042e31-6b5c-49b2-9021-121d1a5455e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:26.089567 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96042e31-6b5c-49b2-9021-121d1a5455e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:26.090323 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96042e31-6b5c-49b2-9021-121d1a5455e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:26.092928 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96042e31-6b5c-49b2-9021-121d1a5455e7 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:26.150710 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1099/2161] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:31:26 2022] POST /identity/v3/projects => generated 375 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:31:26.157255 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9decd8a1-c24c-4354-9cb1-c28677b36146 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:26.172571 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9decd8a1-c24c-4354-9cb1-c28677b36146 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:26.288056 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b9a2ab-9004-4d4b-910e-62c24fe88f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:26.289246 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b9a2ab-9004-4d4b-910e-62c24fe88f2b admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:26.289820 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b9a2ab-9004-4d4b-910e-62c24fe88f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:26.290437 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b9a2ab-9004-4d4b-910e-62c24fe88f2b admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:26.291625 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b9a2ab-9004-4d4b-910e-62c24fe88f2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:26.295163 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b9a2ab-9004-4d4b-910e-62c24fe88f2b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:26.329787 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1063/2162] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:31:26 2022] POST /identity/v3/users => generated 288 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:31:26.335361 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96042e31-6b5c-49b2-9021-121d1a5455e7 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:26.342733 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:26.357107 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96042e31-6b5c-49b2-9021-121d1a5455e7 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:26.364227 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3679e60-9f8a-41af-9edc-07126a4fb3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:26.364927 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3679e60-9f8a-41af-9edc-07126a4fb3c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:26.365210 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3679e60-9f8a-41af-9edc-07126a4fb3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:26.365488 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3679e60-9f8a-41af-9edc-07126a4fb3c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:26.366324 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3679e60-9f8a-41af-9edc-07126a4fb3c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:26.367979 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3679e60-9f8a-41af-9edc-07126a4fb3c2 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:26.383987 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1100/2163] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:31:26.397100 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b9a2ab-9004-4d4b-910e-62c24fe88f2b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:26.417212 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b9a2ab-9004-4d4b-910e-62c24fe88f2b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:26.426578 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f84b60-2aa9-4dcc-8155-00bd5ba2585b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:26.427468 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f84b60-2aa9-4dcc-8155-00bd5ba2585b admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:26.427910 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f84b60-2aa9-4dcc-8155-00bd5ba2585b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:26.428199 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f84b60-2aa9-4dcc-8155-00bd5ba2585b admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:26.429022 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f84b60-2aa9-4dcc-8155-00bd5ba2585b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:26.430790 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f84b60-2aa9-4dcc-8155-00bd5ba2585b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:26.459855 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1064/2164] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:31:26.460481 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3679e60-9f8a-41af-9edc-07126a4fb3c2 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:26.474857 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3679e60-9f8a-41af-9edc-07126a4fb3c2 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:26.483504 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dff35e-3502-4aa4-b8f6-d4aae7edf6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:26.484667 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dff35e-3502-4aa4-b8f6-d4aae7edf6ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:26.485065 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dff35e-3502-4aa4-b8f6-d4aae7edf6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:26.485412 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dff35e-3502-4aa4-b8f6-d4aae7edf6ea admin admin] PATH_INFO: `/v3/projects/416a97d176e548618e83e4c3d57801c9/users/f5f6ae07071844838ffc72c26dc04f24/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:26.512426 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dff35e-3502-4aa4-b8f6-d4aae7edf6ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=416a97d176e548618e83e4c3d57801c9, user_id=f5f6ae07071844838ffc72c26dc04f24, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:26.518249 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dff35e-3502-4aa4-b8f6-d4aae7edf6ea admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:26.542547 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1101/2165] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:31:26 2022] PUT /identity/v3/projects/416a97d176e548618e83e4c3d57801c9/users/f5f6ae07071844838ffc72c26dc04f24/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:26.562371 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d2b19-b2f0-48f3-9b26-33d7969d3f18 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:26.562666 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d2b19-b2f0-48f3-9b26-33d7969d3f18 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:26.562952 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d2b19-b2f0-48f3-9b26-33d7969d3f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:26.610333 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-9b5d2b19-b2f0-48f3-9b26-33d7969d3f18 None None] MFA Rules not processed for user `f5f6ae07071844838ffc72c26dc04f24`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:26.611757 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9b5d2b19-b2f0-48f3-9b26-33d7969d3f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:26.720439 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1065/2166] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:31:26.726734 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dff35e-3502-4aa4-b8f6-d4aae7edf6ea admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:26.862638 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dff35e-3502-4aa4-b8f6-d4aae7edf6ea admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:26.868363 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed356210-b904-46cc-9bcf-dce38434fee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:26.869184 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed356210-b904-46cc-9bcf-dce38434fee6 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:26.869517 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed356210-b904-46cc-9bcf-dce38434fee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:26.869890 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed356210-b904-46cc-9bcf-dce38434fee6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:26.870664 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed356210-b904-46cc-9bcf-dce38434fee6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:26.873093 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed356210-b904-46cc-9bcf-dce38434fee6 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:26.909032 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1102/2167] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:31:26 2022] POST /identity/v3/projects => generated 377 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:31:26.915291 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d2b19-b2f0-48f3-9b26-33d7969d3f18 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:26.935539 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d2b19-b2f0-48f3-9b26-33d7969d3f18 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:27.026669 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ea325-8145-4060-b702-b61f7d98f46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:27.027994 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ea325-8145-4060-b702-b61f7d98f46b admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:27.028776 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ea325-8145-4060-b702-b61f7d98f46b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:27.029400 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ea325-8145-4060-b702-b61f7d98f46b admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:27.030346 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0ea325-8145-4060-b702-b61f7d98f46b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:27.032652 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0ea325-8145-4060-b702-b61f7d98f46b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:27.042579 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:27.070746 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1066/2168] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:31:26 2022] POST /identity/v3/users => generated 289 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:31:27.076774 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed356210-b904-46cc-9bcf-dce38434fee6 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:27.089595 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed356210-b904-46cc-9bcf-dce38434fee6 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:27.095957 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b862611-c7ba-4c17-9cc5-63ea2b45c690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:27.096694 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b862611-c7ba-4c17-9cc5-63ea2b45c690 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:27.097005 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b862611-c7ba-4c17-9cc5-63ea2b45c690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:27.097284 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b862611-c7ba-4c17-9cc5-63ea2b45c690 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:27.098094 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b862611-c7ba-4c17-9cc5-63ea2b45c690 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:27.099806 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b862611-c7ba-4c17-9cc5-63ea2b45c690 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:27.117668 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1103/2169] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:31:27 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:31:27.122658 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ea325-8145-4060-b702-b61f7d98f46b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:27.135079 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ea325-8145-4060-b702-b61f7d98f46b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:27.141290 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd976e0-8c51-4a61-a6a2-3f468693c6b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:27.142225 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd976e0-8c51-4a61-a6a2-3f468693c6b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:27.142548 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd976e0-8c51-4a61-a6a2-3f468693c6b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:27.142826 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd976e0-8c51-4a61-a6a2-3f468693c6b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:27.143617 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd976e0-8c51-4a61-a6a2-3f468693c6b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:27.145303 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd976e0-8c51-4a61-a6a2-3f468693c6b3 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:27.159448 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1067/2170] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:31:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:31:27.165974 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b862611-c7ba-4c17-9cc5-63ea2b45c690 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:27.180983 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b862611-c7ba-4c17-9cc5-63ea2b45c690 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:27.188190 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bdc90c-a559-42f2-afc1-6479fb6782c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:27.189961 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bdc90c-a559-42f2-afc1-6479fb6782c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:27.189961 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bdc90c-a559-42f2-afc1-6479fb6782c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:27.189961 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bdc90c-a559-42f2-afc1-6479fb6782c3 admin admin] PATH_INFO: `/v3/projects/1d86db106a2343b3814a8e2c2e9da83d/users/54cca8b14b94474799a9819a75757fde/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:27.211056 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bdc90c-a559-42f2-afc1-6479fb6782c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d86db106a2343b3814a8e2c2e9da83d, user_id=54cca8b14b94474799a9819a75757fde, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:27.213939 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bdc90c-a559-42f2-afc1-6479fb6782c3 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:27.238690 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1104/2171] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:31:27 2022] PUT /identity/v3/projects/1d86db106a2343b3814a8e2c2e9da83d/users/54cca8b14b94474799a9819a75757fde/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:27.258687 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc899658-9920-4721-b4dd-c56231d2fd49 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:27.259490 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc899658-9920-4721-b4dd-c56231d2fd49 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:27.259807 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc899658-9920-4721-b4dd-c56231d2fd49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:27.296373 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-bc899658-9920-4721-b4dd-c56231d2fd49 None None] MFA Rules not processed for user `54cca8b14b94474799a9819a75757fde`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:27.298201 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-bc899658-9920-4721-b4dd-c56231d2fd49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:27.418196 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1068/2172] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:31:27.424513 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bdc90c-a559-42f2-afc1-6479fb6782c3 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:27.552011 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bdc90c-a559-42f2-afc1-6479fb6782c3 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:27.559697 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3faf5d5-fb7b-439a-b344-87f9e611bd22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:27.560677 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3faf5d5-fb7b-439a-b344-87f9e611bd22 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:27.561194 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3faf5d5-fb7b-439a-b344-87f9e611bd22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:27.561851 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3faf5d5-fb7b-439a-b344-87f9e611bd22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:27.563084 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3faf5d5-fb7b-439a-b344-87f9e611bd22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:27.566997 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3faf5d5-fb7b-439a-b344-87f9e611bd22 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:27.613669 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1105/2173] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:31:27 2022] POST /identity/v3/projects => generated 377 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:31:27.620322 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc899658-9920-4721-b4dd-c56231d2fd49 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:27.640065 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc899658-9920-4721-b4dd-c56231d2fd49 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:27.773045 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50977f1-a57f-4f5e-a8a1-ef267c070fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:27.774144 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50977f1-a57f-4f5e-a8a1-ef267c070fe1 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:27.774818 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50977f1-a57f-4f5e-a8a1-ef267c070fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:27.774818 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50977f1-a57f-4f5e-a8a1-ef267c070fe1 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:27.775474 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50977f1-a57f-4f5e-a8a1-ef267c070fe1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:27.777552 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50977f1-a57f-4f5e-a8a1-ef267c070fe1 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:27.812911 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1069/2174] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:31:27 2022] POST /identity/v3/users => generated 295 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:31:27.817045 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3faf5d5-fb7b-439a-b344-87f9e611bd22 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:27.828945 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:27.844072 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3faf5d5-fb7b-439a-b344-87f9e611bd22 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:27.856716 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa27ee-8430-482d-93a1-7360325d33c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:27.857971 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa27ee-8430-482d-93a1-7360325d33c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:27.858558 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa27ee-8430-482d-93a1-7360325d33c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:27.859172 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa27ee-8430-482d-93a1-7360325d33c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:27.860409 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aa27ee-8430-482d-93a1-7360325d33c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:27.862640 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aa27ee-8430-482d-93a1-7360325d33c9 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:27.877153 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1106/2175] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:31:27 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:31:27.881664 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50977f1-a57f-4f5e-a8a1-ef267c070fe1 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:27.896515 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50977f1-a57f-4f5e-a8a1-ef267c070fe1 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:27.912630 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb751cca-2933-4a35-9b48-257b59534a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:27.914771 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb751cca-2933-4a35-9b48-257b59534a51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:27.915569 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb751cca-2933-4a35-9b48-257b59534a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:27.916319 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb751cca-2933-4a35-9b48-257b59534a51 admin admin] PATH_INFO: `/v3/projects/74e9be8f4440476c97252afdec99242d/users/85c8a477f8a24c208fc202263479c8d7/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:27.941528 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb751cca-2933-4a35-9b48-257b59534a51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74e9be8f4440476c97252afdec99242d, user_id=85c8a477f8a24c208fc202263479c8d7, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:27.945263 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb751cca-2933-4a35-9b48-257b59534a51 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:27.973260 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1070/2176] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:31:27 2022] PUT /identity/v3/projects/74e9be8f4440476c97252afdec99242d/users/85c8a477f8a24c208fc202263479c8d7/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:27.980517 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa27ee-8430-482d-93a1-7360325d33c9 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:28.004205 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa27ee-8430-482d-93a1-7360325d33c9 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:28.146992 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0dba3f-2089-4e32-a5e9-df979b62ed6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:28.147995 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0dba3f-2089-4e32-a5e9-df979b62ed6a admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:28.147995 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0dba3f-2089-4e32-a5e9-df979b62ed6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:28.148375 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0dba3f-2089-4e32-a5e9-df979b62ed6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:28.148974 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0dba3f-2089-4e32-a5e9-df979b62ed6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:28.150772 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0dba3f-2089-4e32-a5e9-df979b62ed6a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:28.179476 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1107/2177] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:31:27 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:31:28.185860 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb751cca-2933-4a35-9b48-257b59534a51 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:28.198849 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb751cca-2933-4a35-9b48-257b59534a51 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:28.204859 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66224abd-6740-496a-8c64-59374dfd471e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:28.205933 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66224abd-6740-496a-8c64-59374dfd471e admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:28.206223 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66224abd-6740-496a-8c64-59374dfd471e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:28.206509 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66224abd-6740-496a-8c64-59374dfd471e admin admin] PATH_INFO: `/v3/projects/74e9be8f4440476c97252afdec99242d/users/85c8a477f8a24c208fc202263479c8d7/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:28.209558 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66224abd-6740-496a-8c64-59374dfd471e admin admin] RBAC: Authorizing `identity:create_grant(project_id=74e9be8f4440476c97252afdec99242d, user_id=85c8a477f8a24c208fc202263479c8d7, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:28.216085 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66224abd-6740-496a-8c64-59374dfd471e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:28.239210 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1071/2178] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:31:28 2022] PUT /identity/v3/projects/74e9be8f4440476c97252afdec99242d/users/85c8a477f8a24c208fc202263479c8d7/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:28.259596 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241ce4d-b449-4051-a6d4-a0416a984eeb None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:28.260102 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241ce4d-b449-4051-a6d4-a0416a984eeb None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:28.260588 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241ce4d-b449-4051-a6d4-a0416a984eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:28.319392 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-5241ce4d-b449-4051-a6d4-a0416a984eeb None None] MFA Rules not processed for user `85c8a477f8a24c208fc202263479c8d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:28.331752 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5241ce4d-b449-4051-a6d4-a0416a984eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:28.517984 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1108/2179] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:28 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:31:28.532369 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66224abd-6740-496a-8c64-59374dfd471e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:28.535009 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-66224abd-6740-496a-8c64-59374dfd471e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:28.596412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:28.690421 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66224abd-6740-496a-8c64-59374dfd471e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:28.698571 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52f13b5-d3d2-4894-a7d6-1fa8364fd02e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:28.699127 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f13b5-d3d2-4894-a7d6-1fa8364fd02e service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:28.700303 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f13b5-d3d2-4894-a7d6-1fa8364fd02e service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:28.700588 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f13b5-d3d2-4894-a7d6-1fa8364fd02e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:28.851525 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52f13b5-d3d2-4894-a7d6-1fa8364fd02e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:28.855030 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52f13b5-d3d2-4894-a7d6-1fa8364fd02e service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:28.875653 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1072/2180] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:31:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:31:31.896878 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5241ce4d-b449-4051-a6d4-a0416a984eeb None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:31.899428 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5241ce4d-b449-4051-a6d4-a0416a984eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:31.923990 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5241ce4d-b449-4051-a6d4-a0416a984eeb None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:31.930369 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c28e99-2c47-40a3-8dad-a40bb32c5698 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:31.931626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c28e99-2c47-40a3-8dad-a40bb32c5698 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:31.932122 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c28e99-2c47-40a3-8dad-a40bb32c5698 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:31.932630 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c28e99-2c47-40a3-8dad-a40bb32c5698 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:32.042981 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c28e99-2c47-40a3-8dad-a40bb32c5698 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:32.045956 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c28e99-2c47-40a3-8dad-a40bb32c5698 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:32.067239 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1109/2181] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:31:31 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:31:32.826366 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52f13b5-d3d2-4894-a7d6-1fa8364fd02e service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:32.828554 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f52f13b5-d3d2-4894-a7d6-1fa8364fd02e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:32.935866 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52f13b5-d3d2-4894-a7d6-1fa8364fd02e service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:32.946429 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f608f624-6715-48f3-8b15-6b966d1607bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:32.947861 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608f624-6715-48f3-8b15-6b966d1607bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:32.948299 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608f624-6715-48f3-8b15-6b966d1607bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:32.948726 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608f624-6715-48f3-8b15-6b966d1607bc admin admin] PATH_INFO: `/v3/users/9350082cd71c4f61b4aec194bc758825` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:32.951094 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f608f624-6715-48f3-8b15-6b966d1607bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9350082cd71c4f61b4aec194bc758825)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:32.953343 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f608f624-6715-48f3-8b15-6b966d1607bc admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:33.037658 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-f608f624-6715-48f3-8b15-6b966d1607bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9350082cd71c4f61b4aec194bc758825'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:33.050109 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-f608f624-6715-48f3-8b15-6b966d1607bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9350082cd71c4f61b4aec194bc758825'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:33.051597 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-f608f624-6715-48f3-8b15-6b966d1607bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9350082cd71c4f61b4aec194bc758825'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:33.052708 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-f608f624-6715-48f3-8b15-6b966d1607bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9350082cd71c4f61b4aec194bc758825'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:33.057569 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c28e99-2c47-40a3-8dad-a40bb32c5698 service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:33.067789 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c28e99-2c47-40a3-8dad-a40bb32c5698 service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:33.072028 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351d52a-0e02-48f4-9e9a-92a7d1571fce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:33.072709 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351d52a-0e02-48f4-9e9a-92a7d1571fce service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:33.072995 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351d52a-0e02-48f4-9e9a-92a7d1571fce service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:33.073272 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351d52a-0e02-48f4-9e9a-92a7d1571fce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:33.075792 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e351d52a-0e02-48f4-9e9a-92a7d1571fce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:33.078371 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-f608f624-6715-48f3-8b15-6b966d1607bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9350082cd71c4f61b4aec194bc758825'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:33.084160 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:33.089999 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e351d52a-0e02-48f4-9e9a-92a7d1571fce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:33.091753 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e351d52a-0e02-48f4-9e9a-92a7d1571fce service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:33.097834 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:33.106456 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1073/2182] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:31:32 2022] DELETE /identity/v3/users/9350082cd71c4f61b4aec194bc758825 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:33.111293 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b9e1a-c7a3-4e9a-afff-93db9b4aa326 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:33.111832 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b9e1a-c7a3-4e9a-afff-93db9b4aa326 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:33.112231 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b9e1a-c7a3-4e9a-afff-93db9b4aa326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:33.112633 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1110/2183] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:31:33 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:31:33.142025 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-649b9e1a-c7a3-4e9a-afff-93db9b4aa326 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:33.144106 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-649b9e1a-c7a3-4e9a-afff-93db9b4aa326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:33.282327 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1074/2184] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:31:33.292368 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351d52a-0e02-48f4-9e9a-92a7d1571fce service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:33.296306 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e351d52a-0e02-48f4-9e9a-92a7d1571fce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:33.481307 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351d52a-0e02-48f4-9e9a-92a7d1571fce service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:33.636821 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55991d7-814a-463a-ac9d-948491ca4b08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:33.639003 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55991d7-814a-463a-ac9d-948491ca4b08 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:33.639796 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55991d7-814a-463a-ac9d-948491ca4b08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:33.640527 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55991d7-814a-463a-ac9d-948491ca4b08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:33.662531 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55991d7-814a-463a-ac9d-948491ca4b08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:33.665991 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55991d7-814a-463a-ac9d-948491ca4b08 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:33.685493 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1111/2185] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:31:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:31:34.261343 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b9e1a-c7a3-4e9a-afff-93db9b4aa326 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:34.265859 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-649b9e1a-c7a3-4e9a-afff-93db9b4aa326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:34.300682 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b9e1a-c7a3-4e9a-afff-93db9b4aa326 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:34.312235 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2742491-bdb7-439b-81f8-c4bb10d16ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:34.313198 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2742491-bdb7-439b-81f8-c4bb10d16ef9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:34.313490 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2742491-bdb7-439b-81f8-c4bb10d16ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:34.313830 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2742491-bdb7-439b-81f8-c4bb10d16ef9 admin admin] PATH_INFO: `/v3/projects/f8127fd1ce2e4b26af08198e49bb6493` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:34.315563 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2742491-bdb7-439b-81f8-c4bb10d16ef9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8127fd1ce2e4b26af08198e49bb6493)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:34.326927 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2742491-bdb7-439b-81f8-c4bb10d16ef9 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:34.466994 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d2742491-bdb7-439b-81f8-c4bb10d16ef9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8127fd1ce2e4b26af08198e49bb6493', 'request_id': 'req-d2742491-bdb7-439b-81f8-c4bb10d16ef9'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:34.488220 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d2742491-bdb7-439b-81f8-c4bb10d16ef9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8127fd1ce2e4b26af08198e49bb6493', 'request_id': 'req-d2742491-bdb7-439b-81f8-c4bb10d16ef9'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:34.511394 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d2742491-bdb7-439b-81f8-c4bb10d16ef9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8127fd1ce2e4b26af08198e49bb6493', 'request_id': 'req-d2742491-bdb7-439b-81f8-c4bb10d16ef9'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:34.547696 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d2742491-bdb7-439b-81f8-c4bb10d16ef9 admin admin] The token cache is being invalidate because project f8127fd1ce2e4b26af08198e49bb6493 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:31:34.548399 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d2742491-bdb7-439b-81f8-c4bb10d16ef9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:34.550227 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d2742491-bdb7-439b-81f8-c4bb10d16ef9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:34.558378 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1075/2186] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:31:34 2022] DELETE /identity/v3/projects/f8127fd1ce2e4b26af08198e49bb6493 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:34.577809 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55991d7-814a-463a-ac9d-948491ca4b08 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:34.610522 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55991d7-814a-463a-ac9d-948491ca4b08 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:34.787689 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f050529-e3e8-4f63-b737-2bd656010080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:34.789625 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f050529-e3e8-4f63-b737-2bd656010080 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:34.797975 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f050529-e3e8-4f63-b737-2bd656010080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:34.798544 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f050529-e3e8-4f63-b737-2bd656010080 admin admin] PATH_INFO: `/v3/users/538e5f8290914151b5d6dfec8c62d27b` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:34.802452 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f050529-e3e8-4f63-b737-2bd656010080 admin admin] RBAC: Authorizing `identity:delete_user(user_id=538e5f8290914151b5d6dfec8c62d27b)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:34.806832 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f050529-e3e8-4f63-b737-2bd656010080 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:34.933235 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0f050529-e3e8-4f63-b737-2bd656010080 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '538e5f8290914151b5d6dfec8c62d27b'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:34.955628 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:34.972709 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0f050529-e3e8-4f63-b737-2bd656010080 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '538e5f8290914151b5d6dfec8c62d27b'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:34.975519 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0f050529-e3e8-4f63-b737-2bd656010080 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '538e5f8290914151b5d6dfec8c62d27b'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:34.977833 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0f050529-e3e8-4f63-b737-2bd656010080 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '538e5f8290914151b5d6dfec8c62d27b'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:34.997825 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0f050529-e3e8-4f63-b737-2bd656010080 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '538e5f8290914151b5d6dfec8c62d27b'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:35.031185 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1112/2187] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:31:34 2022] DELETE /identity/v3/users/538e5f8290914151b5d6dfec8c62d27b => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:35.711720 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2742491-bdb7-439b-81f8-c4bb10d16ef9 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:35.715878 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d2742491-bdb7-439b-81f8-c4bb10d16ef9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:35.743674 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2742491-bdb7-439b-81f8-c4bb10d16ef9 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:35.893012 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2483deb7-ab29-4b16-965e-b4e1b49cc77c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:35.893965 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483deb7-ab29-4b16-965e-b4e1b49cc77c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:35.894250 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483deb7-ab29-4b16-965e-b4e1b49cc77c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:35.894526 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483deb7-ab29-4b16-965e-b4e1b49cc77c admin admin] PATH_INFO: `/v3/projects/2f82e21ec6d5411abbbf04bbf1c4a206` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:35.896553 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2483deb7-ab29-4b16-965e-b4e1b49cc77c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f82e21ec6d5411abbbf04bbf1c4a206)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:35.898878 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2483deb7-ab29-4b16-965e-b4e1b49cc77c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:35.948872 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:36.040369 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2483deb7-ab29-4b16-965e-b4e1b49cc77c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f82e21ec6d5411abbbf04bbf1c4a206', 'request_id': 'req-2483deb7-ab29-4b16-965e-b4e1b49cc77c'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:36.082159 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2483deb7-ab29-4b16-965e-b4e1b49cc77c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f82e21ec6d5411abbbf04bbf1c4a206', 'request_id': 'req-2483deb7-ab29-4b16-965e-b4e1b49cc77c'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:36.102621 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2483deb7-ab29-4b16-965e-b4e1b49cc77c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f82e21ec6d5411abbbf04bbf1c4a206', 'request_id': 'req-2483deb7-ab29-4b16-965e-b4e1b49cc77c'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:36.125726 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2483deb7-ab29-4b16-965e-b4e1b49cc77c admin admin] The token cache is being invalidate because project 2f82e21ec6d5411abbbf04bbf1c4a206 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:31:36.126082 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2483deb7-ab29-4b16-965e-b4e1b49cc77c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:36.126775 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2483deb7-ab29-4b16-965e-b4e1b49cc77c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:36.135282 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1076/2188] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:31:35 2022] DELETE /identity/v3/projects/2f82e21ec6d5411abbbf04bbf1c4a206 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:36.138948 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e22be1-8fa7-4664-b5fc-283c21b72364 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:36.140445 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e22be1-8fa7-4664-b5fc-283c21b72364 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:36.141166 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e22be1-8fa7-4664-b5fc-283c21b72364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:36.173823 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-75e22be1-8fa7-4664-b5fc-283c21b72364 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:36.175676 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-75e22be1-8fa7-4664-b5fc-283c21b72364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:36.320050 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1113/2189] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:31:36.333600 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f287c7-fd47-4fff-8f75-d368883860c7 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:36.334162 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f287c7-fd47-4fff-8f75-d368883860c7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:36.334671 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f287c7-fd47-4fff-8f75-d368883860c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:36.375883 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-99f287c7-fd47-4fff-8f75-d368883860c7 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:36.378038 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-99f287c7-fd47-4fff-8f75-d368883860c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:36.392887 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1077/2190] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:31:36.399968 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e22be1-8fa7-4664-b5fc-283c21b72364 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:36.420334 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e22be1-8fa7-4664-b5fc-283c21b72364 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:36.435487 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56ef829-d6e7-4abc-b078-2c74f57e0273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:36.436896 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ef829-d6e7-4abc-b078-2c74f57e0273 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:36.437422 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ef829-d6e7-4abc-b078-2c74f57e0273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:36.438114 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ef829-d6e7-4abc-b078-2c74f57e0273 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:36.439988 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56ef829-d6e7-4abc-b078-2c74f57e0273 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:31:36.440318 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56ef829-d6e7-4abc-b078-2c74f57e0273 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:36.443647 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56ef829-d6e7-4abc-b078-2c74f57e0273 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:36.471371 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1114/2191] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:31:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:31:36.477720 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f287c7-fd47-4fff-8f75-d368883860c7 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:36.498918 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f287c7-fd47-4fff-8f75-d368883860c7 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:36.508790 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb45a9d-e477-498f-b8f8-bd0bc9a454d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:36.510149 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb45a9d-e477-498f-b8f8-bd0bc9a454d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:36.510964 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb45a9d-e477-498f-b8f8-bd0bc9a454d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:36.512051 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb45a9d-e477-498f-b8f8-bd0bc9a454d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:36.513593 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb45a9d-e477-498f-b8f8-bd0bc9a454d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:36.517518 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb45a9d-e477-498f-b8f8-bd0bc9a454d5 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:36.588599 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1078/2192] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:31:36 2022] POST /identity/v3/projects => generated 375 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:31:36.593694 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56ef829-d6e7-4abc-b078-2c74f57e0273 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:36.615538 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56ef829-d6e7-4abc-b078-2c74f57e0273 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:36.739784 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f92fe-3723-40d3-bb85-265334e34d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:36.740558 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f92fe-3723-40d3-bb85-265334e34d27 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:36.740935 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f92fe-3723-40d3-bb85-265334e34d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:36.741308 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f92fe-3723-40d3-bb85-265334e34d27 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:36.742848 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085f92fe-3723-40d3-bb85-265334e34d27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:36.745092 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085f92fe-3723-40d3-bb85-265334e34d27 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:36.771425 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f701c9-a3c6-48d6-bd72-22372cc98b30 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:36.771990 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f701c9-a3c6-48d6-bd72-22372cc98b30 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:36.771990 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f701c9-a3c6-48d6-bd72-22372cc98b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:36.783072 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1115/2193] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:31:36 2022] POST /identity/v3/users => generated 288 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:31:36.789918 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f92fe-3723-40d3-bb85-265334e34d27 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:36.794240 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-71f701c9-a3c6-48d6-bd72-22372cc98b30 None None] MFA Rules not processed for user `82c323bc7a8c45e39714bb3e7f401c3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:36.795824 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-71f701c9-a3c6-48d6-bd72-22372cc98b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:36.802495 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f92fe-3723-40d3-bb85-265334e34d27 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:36.811465 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c96e4-6fe9-419c-9a77-63d0bd909e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:36.812250 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c96e4-6fe9-419c-9a77-63d0bd909e86 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:36.812528 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c96e4-6fe9-419c-9a77-63d0bd909e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:36.812817 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c96e4-6fe9-419c-9a77-63d0bd909e86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:36.814086 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6c96e4-6fe9-419c-9a77-63d0bd909e86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:36.816136 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6c96e4-6fe9-419c-9a77-63d0bd909e86 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:36.836621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1116/2194] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:31:36.843371 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c96e4-6fe9-419c-9a77-63d0bd909e86 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:36.869706 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c96e4-6fe9-419c-9a77-63d0bd909e86 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:36.873440 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74846a7-9a18-4814-9245-e499dfb08ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:36.875186 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74846a7-9a18-4814-9245-e499dfb08ace admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:36.875916 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74846a7-9a18-4814-9245-e499dfb08ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:36.876618 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74846a7-9a18-4814-9245-e499dfb08ace admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:36.877449 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74846a7-9a18-4814-9245-e499dfb08ace admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:36.879661 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74846a7-9a18-4814-9245-e499dfb08ace admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:36.900874 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1117/2195] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:31:36.901923 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74846a7-9a18-4814-9245-e499dfb08ace admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:36.910473 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1079/2196] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:31:36.916974 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74846a7-9a18-4814-9245-e499dfb08ace admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:36.925525 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce4832-61a1-4b5c-a8fd-8a9ce453f668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:36.926657 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce4832-61a1-4b5c-a8fd-8a9ce453f668 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:36.926956 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce4832-61a1-4b5c-a8fd-8a9ce453f668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:36.927546 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce4832-61a1-4b5c-a8fd-8a9ce453f668 admin admin] PATH_INFO: `/v3/projects/bd4546e76587483cba926040be2b5832/users/a9e96baff8144ef18eb608bc94404719/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:36.957771 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ce4832-61a1-4b5c-a8fd-8a9ce453f668 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd4546e76587483cba926040be2b5832, user_id=a9e96baff8144ef18eb608bc94404719, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:36.964758 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ce4832-61a1-4b5c-a8fd-8a9ce453f668 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:37.013687 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1118/2197] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:31:36 2022] PUT /identity/v3/projects/bd4546e76587483cba926040be2b5832/users/a9e96baff8144ef18eb608bc94404719/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:37.024950 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e5123d-30a9-4939-b5c2-3d0642b78fca None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:37.025236 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e5123d-30a9-4939-b5c2-3d0642b78fca None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:37.025944 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e5123d-30a9-4939-b5c2-3d0642b78fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:37.085244 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-20e5123d-30a9-4939-b5c2-3d0642b78fca None None] MFA Rules not processed for user `a9e96baff8144ef18eb608bc94404719`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:37.086686 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-20e5123d-30a9-4939-b5c2-3d0642b78fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:37.277492 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1080/2198] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:37 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:31:37.284454 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce4832-61a1-4b5c-a8fd-8a9ce453f668 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:37.302255 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:37.438004 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce4832-61a1-4b5c-a8fd-8a9ce453f668 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:37.447771 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e90667-2204-4787-8343-2d0e60b09cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:37.449039 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e90667-2204-4787-8343-2d0e60b09cc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:37.451424 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e90667-2204-4787-8343-2d0e60b09cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:37.451424 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e90667-2204-4787-8343-2d0e60b09cc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:37.451797 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e90667-2204-4787-8343-2d0e60b09cc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:37.458039 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e90667-2204-4787-8343-2d0e60b09cc6 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:37.516058 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1119/2199] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:31:37 2022] POST /identity/v3/projects => generated 375 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:31:37.518892 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e5123d-30a9-4939-b5c2-3d0642b78fca None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:37.542294 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e5123d-30a9-4939-b5c2-3d0642b78fca None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:37.562644 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:37.673154 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2600e2-a716-4733-8c53-068076c17e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:37.674260 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2600e2-a716-4733-8c53-068076c17e3e admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:37.674634 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2600e2-a716-4733-8c53-068076c17e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:37.674919 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2600e2-a716-4733-8c53-068076c17e3e admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:37.675869 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2600e2-a716-4733-8c53-068076c17e3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:37.681022 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2600e2-a716-4733-8c53-068076c17e3e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:37.718446 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1081/2200] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:31:37 2022] POST /identity/v3/users => generated 294 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:31:37.722908 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e90667-2204-4787-8343-2d0e60b09cc6 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:37.759358 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e90667-2204-4787-8343-2d0e60b09cc6 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:37.787194 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d234f158-ae3f-496b-951d-0e4e823c5ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:37.788540 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234f158-ae3f-496b-951d-0e4e823c5ebe admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:37.789180 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234f158-ae3f-496b-951d-0e4e823c5ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:37.789857 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234f158-ae3f-496b-951d-0e4e823c5ebe admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:37.792054 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d234f158-ae3f-496b-951d-0e4e823c5ebe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:37.796490 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d234f158-ae3f-496b-951d-0e4e823c5ebe admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:37.851960 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1120/2201] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:31:37 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:31:37.858745 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2600e2-a716-4733-8c53-068076c17e3e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:37.880936 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d234f158-ae3f-496b-951d-0e4e823c5ebe admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:37.884810 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d234f158-ae3f-496b-951d-0e4e823c5ebe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:37.892451 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2600e2-a716-4733-8c53-068076c17e3e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:37.909022 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d234f158-ae3f-496b-951d-0e4e823c5ebe admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:37.914817 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca5e0f4-763f-481f-92de-b6031a2c3f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:37.917609 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd4a2a6-4ac0-4be6-bd3f-357855b1c7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:37.918365 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca5e0f4-763f-481f-92de-b6031a2c3f91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:37.919017 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca5e0f4-763f-481f-92de-b6031a2c3f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:37.919371 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4a2a6-4ac0-4be6-bd3f-357855b1c7e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:37.919824 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4a2a6-4ac0-4be6-bd3f-357855b1c7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:37.920290 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca5e0f4-763f-481f-92de-b6031a2c3f91 admin admin] PATH_INFO: `/v3/projects/7c40fb6c011f47bdb8f89fdb56d88e69/users/04ecdb2067df4962aef6a6a7ad8cbd61/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:37.920652 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4a2a6-4ac0-4be6-bd3f-357855b1c7e1 admin admin] PATH_INFO: `/v3/users/f5f6ae07071844838ffc72c26dc04f24` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:37.923009 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd4a2a6-4ac0-4be6-bd3f-357855b1c7e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5f6ae07071844838ffc72c26dc04f24)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:37.925414 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd4a2a6-4ac0-4be6-bd3f-357855b1c7e1 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:37.960912 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca5e0f4-763f-481f-92de-b6031a2c3f91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c40fb6c011f47bdb8f89fdb56d88e69, user_id=04ecdb2067df4962aef6a6a7ad8cbd61, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:37.964631 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca5e0f4-763f-481f-92de-b6031a2c3f91 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:37.990508 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1082/2202] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:31:37 2022] PUT /identity/v3/projects/7c40fb6c011f47bdb8f89fdb56d88e69/users/04ecdb2067df4962aef6a6a7ad8cbd61/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:38.008619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca5e0f4-763f-481f-92de-b6031a2c3f91 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:38.083252 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5dd4a2a6-4ac0-4be6-bd3f-357855b1c7e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5f6ae07071844838ffc72c26dc04f24'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:38.142533 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5dd4a2a6-4ac0-4be6-bd3f-357855b1c7e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5f6ae07071844838ffc72c26dc04f24'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:38.145142 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5dd4a2a6-4ac0-4be6-bd3f-357855b1c7e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5f6ae07071844838ffc72c26dc04f24'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:38.146439 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5dd4a2a6-4ac0-4be6-bd3f-357855b1c7e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5f6ae07071844838ffc72c26dc04f24'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:38.171809 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5dd4a2a6-4ac0-4be6-bd3f-357855b1c7e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5f6ae07071844838ffc72c26dc04f24'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:38.206998 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca5e0f4-763f-481f-92de-b6031a2c3f91 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:38.226910 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1121/2203] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:31:37 2022] DELETE /identity/v3/users/f5f6ae07071844838ffc72c26dc04f24 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:38.230694 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d560a9-efb9-41f0-9b6a-5faf60ecdbaf None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:38.231903 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d560a9-efb9-41f0-9b6a-5faf60ecdbaf None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:38.232409 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d560a9-efb9-41f0-9b6a-5faf60ecdbaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:38.245088 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb5463f-f978-45c5-8a26-8c3b88212015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:38.246475 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb5463f-f978-45c5-8a26-8c3b88212015 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:38.251483 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb5463f-f978-45c5-8a26-8c3b88212015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:38.253019 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb5463f-f978-45c5-8a26-8c3b88212015 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:38.261307 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb5463f-f978-45c5-8a26-8c3b88212015 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:38.266012 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb5463f-f978-45c5-8a26-8c3b88212015 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:38.278508 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-d3d560a9-efb9-41f0-9b6a-5faf60ecdbaf None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:38.281533 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d3d560a9-efb9-41f0-9b6a-5faf60ecdbaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:38.311319 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1083/2204] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:31:38 2022] GET /identity/v3/roles => generated 1420 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:31:38.315985 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb5463f-f978-45c5-8a26-8c3b88212015 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:38.321201 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-cbb5463f-f978-45c5-8a26-8c3b88212015 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:38.386423 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb5463f-f978-45c5-8a26-8c3b88212015 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:38.422608 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:38.520294 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1122/2205] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:31:38.534999 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d560a9-efb9-41f0-9b6a-5faf60ecdbaf None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:38.537885 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d3d560a9-efb9-41f0-9b6a-5faf60ecdbaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:38.575316 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7abecaa-6631-4bf5-9213-e208471ca993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:38.577701 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7abecaa-6631-4bf5-9213-e208471ca993 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:38.577701 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7abecaa-6631-4bf5-9213-e208471ca993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:38.587415 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7abecaa-6631-4bf5-9213-e208471ca993 admin admin] PATH_INFO: `/v3/projects/7c40fb6c011f47bdb8f89fdb56d88e69/users/04ecdb2067df4962aef6a6a7ad8cbd61/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:38.590812 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7abecaa-6631-4bf5-9213-e208471ca993 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c40fb6c011f47bdb8f89fdb56d88e69, user_id=04ecdb2067df4962aef6a6a7ad8cbd61, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:38.594601 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7abecaa-6631-4bf5-9213-e208471ca993 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:38.621238 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1084/2206] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:31:38 2022] PUT /identity/v3/projects/7c40fb6c011f47bdb8f89fdb56d88e69/users/04ecdb2067df4962aef6a6a7ad8cbd61/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:38.636482 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60a188-3ef5-4821-909a-d6974ad21310 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:38.636769 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60a188-3ef5-4821-909a-d6974ad21310 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:38.638338 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60a188-3ef5-4821-909a-d6974ad21310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:38.684422 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:38.703606 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d560a9-efb9-41f0-9b6a-5faf60ecdbaf None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:38.709442 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-bd60a188-3ef5-4821-909a-d6974ad21310 None None] MFA Rules not processed for user `04ecdb2067df4962aef6a6a7ad8cbd61`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:38.712544 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-bd60a188-3ef5-4821-909a-d6974ad21310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:38.863401 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cf15a1-520e-4fb3-b2ef-49d5ec9b79aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:38.865016 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf15a1-520e-4fb3-b2ef-49d5ec9b79aa service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:38.865681 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf15a1-520e-4fb3-b2ef-49d5ec9b79aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:38.866505 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf15a1-520e-4fb3-b2ef-49d5ec9b79aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:38.940648 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1085/2207] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:38 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:31:38.961959 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd60a188-3ef5-4821-909a-d6974ad21310 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:38.964680 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-bd60a188-3ef5-4821-909a-d6974ad21310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:39.071558 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cf15a1-520e-4fb3-b2ef-49d5ec9b79aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:39.076919 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cf15a1-520e-4fb3-b2ef-49d5ec9b79aa service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:39.101217 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1123/2208] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:31:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:31:39.152403 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd60a188-3ef5-4821-909a-d6974ad21310 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:39.165355 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f170ec-9bce-46b7-a57f-14ad96377c28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:39.166257 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f170ec-9bce-46b7-a57f-14ad96377c28 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:39.167475 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f170ec-9bce-46b7-a57f-14ad96377c28 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:39.167475 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f170ec-9bce-46b7-a57f-14ad96377c28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:39.172866 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b8f170ec-9bce-46b7-a57f-14ad96377c28 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:39.373833 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f170ec-9bce-46b7-a57f-14ad96377c28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:39.376765 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f170ec-9bce-46b7-a57f-14ad96377c28 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:39.399475 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1086/2209] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:31:38 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:31:39.771535 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cf15a1-520e-4fb3-b2ef-49d5ec9b79aa service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:39.774809 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-12cf15a1-520e-4fb3-b2ef-49d5ec9b79aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:39.805248 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cf15a1-520e-4fb3-b2ef-49d5ec9b79aa service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:39.819149 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f7af78-d88a-41bf-a6ab-45594b9202da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:39.821205 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f7af78-d88a-41bf-a6ab-45594b9202da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:39.821803 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f7af78-d88a-41bf-a6ab-45594b9202da admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:39.822380 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f7af78-d88a-41bf-a6ab-45594b9202da admin admin] PATH_INFO: `/v3/projects/416a97d176e548618e83e4c3d57801c9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:39.830341 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f7af78-d88a-41bf-a6ab-45594b9202da admin admin] RBAC: Authorizing `identity:delete_project(project_id=416a97d176e548618e83e4c3d57801c9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:39.839206 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f7af78-d88a-41bf-a6ab-45594b9202da admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:39.991756 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-86f7af78-d88a-41bf-a6ab-45594b9202da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '416a97d176e548618e83e4c3d57801c9', 'request_id': 'req-86f7af78-d88a-41bf-a6ab-45594b9202da'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:39.999985 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:40.002758 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-86f7af78-d88a-41bf-a6ab-45594b9202da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '416a97d176e548618e83e4c3d57801c9', 'request_id': 'req-86f7af78-d88a-41bf-a6ab-45594b9202da'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:40.019136 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-86f7af78-d88a-41bf-a6ab-45594b9202da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '416a97d176e548618e83e4c3d57801c9', 'request_id': 'req-86f7af78-d88a-41bf-a6ab-45594b9202da'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:40.118276 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-86f7af78-d88a-41bf-a6ab-45594b9202da admin admin] The token cache is being invalidate because project 416a97d176e548618e83e4c3d57801c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:31:40.118799 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-86f7af78-d88a-41bf-a6ab-45594b9202da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:40.128543 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-86f7af78-d88a-41bf-a6ab-45594b9202da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:40.134357 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1124/2210] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:31:39 2022] DELETE /identity/v3/projects/416a97d176e548618e83e4c3d57801c9 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:40.143980 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f170ec-9bce-46b7-a57f-14ad96377c28 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:40.211798 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f170ec-9bce-46b7-a57f-14ad96377c28 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:40.379126 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:40.402182 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd928989-cffd-4268-84e1-432c6987afed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:40.403874 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd928989-cffd-4268-84e1-432c6987afed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:40.404448 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd928989-cffd-4268-84e1-432c6987afed admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:40.404959 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd928989-cffd-4268-84e1-432c6987afed admin admin] PATH_INFO: `/v3/users/54cca8b14b94474799a9819a75757fde` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:40.406611 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd928989-cffd-4268-84e1-432c6987afed admin admin] RBAC: Authorizing `identity:delete_user(user_id=54cca8b14b94474799a9819a75757fde)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:40.410176 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd928989-cffd-4268-84e1-432c6987afed admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:40.551823 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-cd928989-cffd-4268-84e1-432c6987afed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54cca8b14b94474799a9819a75757fde'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:40.570730 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-cd928989-cffd-4268-84e1-432c6987afed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54cca8b14b94474799a9819a75757fde'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:40.572169 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-cd928989-cffd-4268-84e1-432c6987afed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54cca8b14b94474799a9819a75757fde'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:40.574114 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-cd928989-cffd-4268-84e1-432c6987afed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54cca8b14b94474799a9819a75757fde'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:40.604642 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-cd928989-cffd-4268-84e1-432c6987afed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54cca8b14b94474799a9819a75757fde'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:40.647052 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1087/2211] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:31:40 2022] DELETE /identity/v3/users/54cca8b14b94474799a9819a75757fde => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:41.920636 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f7af78-d88a-41bf-a6ab-45594b9202da admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:41.924709 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-86f7af78-d88a-41bf-a6ab-45594b9202da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:41.957889 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f7af78-d88a-41bf-a6ab-45594b9202da admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:42.241681 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703ce10f-5851-4d60-affc-93c4b25061a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:42.243871 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ce10f-5851-4d60-affc-93c4b25061a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:42.244526 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ce10f-5851-4d60-affc-93c4b25061a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:42.245139 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ce10f-5851-4d60-affc-93c4b25061a5 admin admin] PATH_INFO: `/v3/projects/1d86db106a2343b3814a8e2c2e9da83d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:42.250228 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703ce10f-5851-4d60-affc-93c4b25061a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d86db106a2343b3814a8e2c2e9da83d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:42.257206 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703ce10f-5851-4d60-affc-93c4b25061a5 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:42.401559 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-703ce10f-5851-4d60-affc-93c4b25061a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d86db106a2343b3814a8e2c2e9da83d', 'request_id': 'req-703ce10f-5851-4d60-affc-93c4b25061a5'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:42.410903 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-703ce10f-5851-4d60-affc-93c4b25061a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d86db106a2343b3814a8e2c2e9da83d', 'request_id': 'req-703ce10f-5851-4d60-affc-93c4b25061a5'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:42.443438 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-703ce10f-5851-4d60-affc-93c4b25061a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d86db106a2343b3814a8e2c2e9da83d', 'request_id': 'req-703ce10f-5851-4d60-affc-93c4b25061a5'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:42.477968 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-703ce10f-5851-4d60-affc-93c4b25061a5 admin admin] The token cache is being invalidate because project 1d86db106a2343b3814a8e2c2e9da83d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:31:42.478478 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-703ce10f-5851-4d60-affc-93c4b25061a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:42.480089 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-703ce10f-5851-4d60-affc-93c4b25061a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:42.491795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1125/2212] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:31:41 2022] DELETE /identity/v3/projects/1d86db106a2343b3814a8e2c2e9da83d => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:42.498260 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd928989-cffd-4268-84e1-432c6987afed admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:42.500682 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-cd928989-cffd-4268-84e1-432c6987afed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:42.555424 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd928989-cffd-4268-84e1-432c6987afed admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:42.700142 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be98498-cc5b-44d8-9918-edcc55f2ddd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:42.700625 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be98498-cc5b-44d8-9918-edcc55f2ddd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:42.701666 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be98498-cc5b-44d8-9918-edcc55f2ddd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:42.701836 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be98498-cc5b-44d8-9918-edcc55f2ddd9 admin admin] PATH_INFO: `/v3/users/85c8a477f8a24c208fc202263479c8d7` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:42.705254 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be98498-cc5b-44d8-9918-edcc55f2ddd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85c8a477f8a24c208fc202263479c8d7)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:42.708596 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be98498-cc5b-44d8-9918-edcc55f2ddd9 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:42.805385 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3be98498-cc5b-44d8-9918-edcc55f2ddd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85c8a477f8a24c208fc202263479c8d7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:42.826546 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3be98498-cc5b-44d8-9918-edcc55f2ddd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85c8a477f8a24c208fc202263479c8d7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:42.829670 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3be98498-cc5b-44d8-9918-edcc55f2ddd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85c8a477f8a24c208fc202263479c8d7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:42.831336 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3be98498-cc5b-44d8-9918-edcc55f2ddd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85c8a477f8a24c208fc202263479c8d7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:42.861904 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:42.866967 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3be98498-cc5b-44d8-9918-edcc55f2ddd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85c8a477f8a24c208fc202263479c8d7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:42.901454 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1088/2213] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:31:42 2022] DELETE /identity/v3/users/85c8a477f8a24c208fc202263479c8d7 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:31:43.415465 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703ce10f-5851-4d60-affc-93c4b25061a5 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:43.420781 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-703ce10f-5851-4d60-affc-93c4b25061a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:43.444754 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703ce10f-5851-4d60-affc-93c4b25061a5 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:43.561091 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc8953a-8f0b-4d3b-8713-b6e5d331d35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:43.562753 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8953a-8f0b-4d3b-8713-b6e5d331d35d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:43.563242 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8953a-8f0b-4d3b-8713-b6e5d331d35d admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:43.563712 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8953a-8f0b-4d3b-8713-b6e5d331d35d admin admin] PATH_INFO: `/v3/projects/74e9be8f4440476c97252afdec99242d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:43.566230 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc8953a-8f0b-4d3b-8713-b6e5d331d35d admin admin] RBAC: Authorizing `identity:delete_project(project_id=74e9be8f4440476c97252afdec99242d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:43.569777 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc8953a-8f0b-4d3b-8713-b6e5d331d35d admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:43.600703 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:43.687750 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8cc8953a-8f0b-4d3b-8713-b6e5d331d35d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74e9be8f4440476c97252afdec99242d', 'request_id': 'req-8cc8953a-8f0b-4d3b-8713-b6e5d331d35d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:43.704011 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8cc8953a-8f0b-4d3b-8713-b6e5d331d35d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74e9be8f4440476c97252afdec99242d', 'request_id': 'req-8cc8953a-8f0b-4d3b-8713-b6e5d331d35d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:43.720031 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8cc8953a-8f0b-4d3b-8713-b6e5d331d35d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74e9be8f4440476c97252afdec99242d', 'request_id': 'req-8cc8953a-8f0b-4d3b-8713-b6e5d331d35d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:43.753323 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8cc8953a-8f0b-4d3b-8713-b6e5d331d35d admin admin] The token cache is being invalidate because project 74e9be8f4440476c97252afdec99242d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:31:43.753605 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8cc8953a-8f0b-4d3b-8713-b6e5d331d35d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:43.756867 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8cc8953a-8f0b-4d3b-8713-b6e5d331d35d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:43.758115 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1126/2214] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:31:43 2022] DELETE /identity/v3/projects/74e9be8f4440476c97252afdec99242d => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:43.766822 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9965f4e-d12a-4052-8d9c-5b677a0a7af0 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:43.767667 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9965f4e-d12a-4052-8d9c-5b677a0a7af0 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:43.767923 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9965f4e-d12a-4052-8d9c-5b677a0a7af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:43.829606 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b9965f4e-d12a-4052-8d9c-5b677a0a7af0 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:43.834606 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b9965f4e-d12a-4052-8d9c-5b677a0a7af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:43.999531 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1089/2215] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:31:44.014777 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f7307-f7b5-49c4-aa5b-d089cc9d741e None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:44.015268 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f7307-f7b5-49c4-aa5b-d089cc9d741e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:44.015782 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f7307-f7b5-49c4-aa5b-d089cc9d741e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:44.052371 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-cc1f7307-f7b5-49c4-aa5b-d089cc9d741e None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:44.053838 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-cc1f7307-f7b5-49c4-aa5b-d089cc9d741e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:44.063965 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1127/2216] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:31:44.073122 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9965f4e-d12a-4052-8d9c-5b677a0a7af0 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:44.088007 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9965f4e-d12a-4052-8d9c-5b677a0a7af0 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:44.095148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15493f41-acec-480d-9751-29b5b69c1219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:44.096321 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15493f41-acec-480d-9751-29b5b69c1219 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:44.096321 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15493f41-acec-480d-9751-29b5b69c1219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:44.096321 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15493f41-acec-480d-9751-29b5b69c1219 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:44.097230 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15493f41-acec-480d-9751-29b5b69c1219 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:31:44.097801 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15493f41-acec-480d-9751-29b5b69c1219 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:44.099356 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15493f41-acec-480d-9751-29b5b69c1219 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:44.118859 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1090/2217] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:31:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:31:44.127369 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f7307-f7b5-49c4-aa5b-d089cc9d741e None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:44.152297 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f7307-f7b5-49c4-aa5b-d089cc9d741e None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:44.161310 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba87a01e-74bb-437e-906b-5b024d0df641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:44.164192 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba87a01e-74bb-437e-906b-5b024d0df641 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:44.164686 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba87a01e-74bb-437e-906b-5b024d0df641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:44.165152 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba87a01e-74bb-437e-906b-5b024d0df641 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:44.173610 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba87a01e-74bb-437e-906b-5b024d0df641 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:44.173610 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba87a01e-74bb-437e-906b-5b024d0df641 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:44.222853 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1128/2218] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:31:44 2022] POST /identity/v3/projects => generated 369 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:31:44.231380 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15493f41-acec-480d-9751-29b5b69c1219 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:44.249220 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15493f41-acec-480d-9751-29b5b69c1219 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:44.359547 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f8344b-3874-425a-b1c2-80ab212a6320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:44.360942 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f8344b-3874-425a-b1c2-80ab212a6320 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:44.361528 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f8344b-3874-425a-b1c2-80ab212a6320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:44.362483 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f8344b-3874-425a-b1c2-80ab212a6320 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:44.363848 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f8344b-3874-425a-b1c2-80ab212a6320 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:44.367406 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f8344b-3874-425a-b1c2-80ab212a6320 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:44.406690 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1091/2219] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:31:44 2022] POST /identity/v3/users => generated 285 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:31:44.414092 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba87a01e-74bb-437e-906b-5b024d0df641 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:44.434344 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba87a01e-74bb-437e-906b-5b024d0df641 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:44.447911 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8334136a-9bcb-4671-a2c8-bc36d1edf173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:44.449753 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334136a-9bcb-4671-a2c8-bc36d1edf173 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:44.450271 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334136a-9bcb-4671-a2c8-bc36d1edf173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:44.450816 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334136a-9bcb-4671-a2c8-bc36d1edf173 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:44.452161 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8334136a-9bcb-4671-a2c8-bc36d1edf173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:44.455485 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8334136a-9bcb-4671-a2c8-bc36d1edf173 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:44.474023 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1129/2220] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:31:44 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:31:44.480788 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f8344b-3874-425a-b1c2-80ab212a6320 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:44.502039 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f8344b-3874-425a-b1c2-80ab212a6320 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:44.511495 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e842c97-b374-4e35-ae7b-f6fd413dcfdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:44.512905 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e842c97-b374-4e35-ae7b-f6fd413dcfdd admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:44.513536 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e842c97-b374-4e35-ae7b-f6fd413dcfdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:44.514044 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e842c97-b374-4e35-ae7b-f6fd413dcfdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:44.515593 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e842c97-b374-4e35-ae7b-f6fd413dcfdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:44.518992 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e842c97-b374-4e35-ae7b-f6fd413dcfdd admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:44.540287 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1092/2221] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:31:44 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:31:44.547629 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8334136a-9bcb-4671-a2c8-bc36d1edf173 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:44.565484 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8334136a-9bcb-4671-a2c8-bc36d1edf173 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:44.576494 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a84174-da5d-4dcd-94db-71d2ada869a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:44.578242 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a84174-da5d-4dcd-94db-71d2ada869a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:44.578766 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a84174-da5d-4dcd-94db-71d2ada869a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:44.579290 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a84174-da5d-4dcd-94db-71d2ada869a2 admin admin] PATH_INFO: `/v3/projects/1062e366f2224ea380ef02b5a61bfcd5/users/8be1b8289d7548aca203e02ddc8808bb/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:44.600224 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a84174-da5d-4dcd-94db-71d2ada869a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1062e366f2224ea380ef02b5a61bfcd5, user_id=8be1b8289d7548aca203e02ddc8808bb, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:44.603226 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a84174-da5d-4dcd-94db-71d2ada869a2 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:44.623830 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1130/2222] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:31:44 2022] PUT /identity/v3/projects/1062e366f2224ea380ef02b5a61bfcd5/users/8be1b8289d7548aca203e02ddc8808bb/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:44.636722 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec9801-d420-480f-af93-092ffd657835 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:44.637305 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec9801-d420-480f-af93-092ffd657835 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:44.637956 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec9801-d420-480f-af93-092ffd657835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:44.681680 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-97ec9801-d420-480f-af93-092ffd657835 None None] MFA Rules not processed for user `8be1b8289d7548aca203e02ddc8808bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:44.682952 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-97ec9801-d420-480f-af93-092ffd657835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:44.751595 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:44.796315 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1093/2223] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:44 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:31:44.821104 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a84174-da5d-4dcd-94db-71d2ada869a2 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:44.823944 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-33a84174-da5d-4dcd-94db-71d2ada869a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:44.919012 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7847d-aa1d-458a-8478-ed672ba36c67 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:44.920198 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7847d-aa1d-458a-8478-ed672ba36c67 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:44.921980 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7847d-aa1d-458a-8478-ed672ba36c67 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:44.925982 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1094/2224] 10.222.0.87 () {54 vars in 921 bytes} [Tue Jun 21 18:31:44 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:31:44.941117 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b20992-3ac5-42fd-be7a-bc5c698a4f2a None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:44.941815 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b20992-3ac5-42fd-be7a-bc5c698a4f2a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:44.942284 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b20992-3ac5-42fd-be7a-bc5c698a4f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:44.988467 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-05b20992-3ac5-42fd-be7a-bc5c698a4f2a None None] MFA Rules not processed for user `ef1d49b36cdc4665b82487be97e9dcd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:44.990650 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-05b20992-3ac5-42fd-be7a-bc5c698a4f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:45.018571 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a84174-da5d-4dcd-94db-71d2ada869a2 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:45.026587 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302e0e91-cf50-40a6-a9d3-91f55467d6b8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7da4bf6f3be47b4aa25c6240cb4be03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:45.028607 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e0e91-cf50-40a6-a9d3-91f55467d6b8 service glance] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:45.029285 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e0e91-cf50-40a6-a9d3-91f55467d6b8 service glance] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:45.029847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e0e91-cf50-40a6-a9d3-91f55467d6b8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:45.048381 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302e0e91-cf50-40a6-a9d3-91f55467d6b8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:45.051388 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302e0e91-cf50-40a6-a9d3-91f55467d6b8 service glance] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:45.076861 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1131/2225] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:31:44 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:31:45.210163 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1095/2226] 10.222.0.87 () {58 vars in 1018 bytes} [Tue Jun 21 18:31:44 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:31:45.469514 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302e0e91-cf50-40a6-a9d3-91f55467d6b8 service glance] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:45.471679 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-302e0e91-cf50-40a6-a9d3-91f55467d6b8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:45.511326 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302e0e91-cf50-40a6-a9d3-91f55467d6b8 service glance] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:45.523884 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca86dfd-5118-4b37-b626-e354293ddf46 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef1d49b36cdc4665b82487be97e9dcd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:45.524816 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca86dfd-5118-4b37-b626-e354293ddf46 service placement] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:45.525213 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca86dfd-5118-4b37-b626-e354293ddf46 service placement] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:45.525658 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca86dfd-5118-4b37-b626-e354293ddf46 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:45.547693 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca86dfd-5118-4b37-b626-e354293ddf46 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:45.550325 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca86dfd-5118-4b37-b626-e354293ddf46 service placement] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:45.570271 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1132/2227] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:31:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:31:46.619785 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b20992-3ac5-42fd-be7a-bc5c698a4f2a None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:46.622964 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-05b20992-3ac5-42fd-be7a-bc5c698a4f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:46.771476 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b20992-3ac5-42fd-be7a-bc5c698a4f2a None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:46.780197 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce44f7-4251-4f9c-aaf5-813d1958c5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:46.781665 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce44f7-4251-4f9c-aaf5-813d1958c5b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:46.783667 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce44f7-4251-4f9c-aaf5-813d1958c5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:46.783667 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce44f7-4251-4f9c-aaf5-813d1958c5b3 admin admin] PATH_INFO: `/v3/users/e60703da011b4ed6b35f19f9ea4cafe7` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:46.784571 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cce44f7-4251-4f9c-aaf5-813d1958c5b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e60703da011b4ed6b35f19f9ea4cafe7)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:46.787084 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cce44f7-4251-4f9c-aaf5-813d1958c5b3 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:46.886249 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6cce44f7-4251-4f9c-aaf5-813d1958c5b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e60703da011b4ed6b35f19f9ea4cafe7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:46.899601 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6cce44f7-4251-4f9c-aaf5-813d1958c5b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e60703da011b4ed6b35f19f9ea4cafe7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:46.900744 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6cce44f7-4251-4f9c-aaf5-813d1958c5b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e60703da011b4ed6b35f19f9ea4cafe7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:46.902782 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6cce44f7-4251-4f9c-aaf5-813d1958c5b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e60703da011b4ed6b35f19f9ea4cafe7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:46.936962 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6cce44f7-4251-4f9c-aaf5-813d1958c5b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e60703da011b4ed6b35f19f9ea4cafe7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:46.957545 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:46.967881 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1096/2228] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:31:46 2022] DELETE /identity/v3/users/e60703da011b4ed6b35f19f9ea4cafe7 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:31:47.463283 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca86dfd-5118-4b37-b626-e354293ddf46 service placement] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:47.496896 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca86dfd-5118-4b37-b626-e354293ddf46 service placement] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:47.519035 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:47.660059 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6e8f2f-f288-4495-aebb-7ffd0d0148e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:47.662042 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e8f2f-f288-4495-aebb-7ffd0d0148e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:47.662667 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e8f2f-f288-4495-aebb-7ffd0d0148e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:47.663211 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e8f2f-f288-4495-aebb-7ffd0d0148e4 admin admin] PATH_INFO: `/v3/projects/97e3300702ff4ec9a415f1d4d3269499` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:47.666112 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6e8f2f-f288-4495-aebb-7ffd0d0148e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97e3300702ff4ec9a415f1d4d3269499)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:47.670248 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6e8f2f-f288-4495-aebb-7ffd0d0148e4 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:47.768544 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-af6e8f2f-f288-4495-aebb-7ffd0d0148e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97e3300702ff4ec9a415f1d4d3269499', 'request_id': 'req-af6e8f2f-f288-4495-aebb-7ffd0d0148e4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:47.776653 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-af6e8f2f-f288-4495-aebb-7ffd0d0148e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97e3300702ff4ec9a415f1d4d3269499', 'request_id': 'req-af6e8f2f-f288-4495-aebb-7ffd0d0148e4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:47.788022 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-af6e8f2f-f288-4495-aebb-7ffd0d0148e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97e3300702ff4ec9a415f1d4d3269499', 'request_id': 'req-af6e8f2f-f288-4495-aebb-7ffd0d0148e4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:47.806114 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-af6e8f2f-f288-4495-aebb-7ffd0d0148e4 admin admin] The token cache is being invalidate because project 97e3300702ff4ec9a415f1d4d3269499 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:31:47.806615 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-af6e8f2f-f288-4495-aebb-7ffd0d0148e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:47.807616 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-af6e8f2f-f288-4495-aebb-7ffd0d0148e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:47.813858 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1133/2229] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:31:47 2022] DELETE /identity/v3/projects/97e3300702ff4ec9a415f1d4d3269499 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:31:47.815418 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce44f7-4251-4f9c-aaf5-813d1958c5b3 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:47.817078 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6cce44f7-4251-4f9c-aaf5-813d1958c5b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:47.838470 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce44f7-4251-4f9c-aaf5-813d1958c5b3 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:47.939982 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004cb69d-021d-417a-84c0-8ef096d1687b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:47.941514 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004cb69d-021d-417a-84c0-8ef096d1687b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:47.942047 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004cb69d-021d-417a-84c0-8ef096d1687b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:47.942520 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004cb69d-021d-417a-84c0-8ef096d1687b admin admin] PATH_INFO: `/v3/users/e7e129cf70ab46418e83e7e7488673bb` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:47.944652 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004cb69d-021d-417a-84c0-8ef096d1687b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7e129cf70ab46418e83e7e7488673bb)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:47.949246 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004cb69d-021d-417a-84c0-8ef096d1687b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:48.040281 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-004cb69d-021d-417a-84c0-8ef096d1687b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7e129cf70ab46418e83e7e7488673bb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:48.054386 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-004cb69d-021d-417a-84c0-8ef096d1687b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7e129cf70ab46418e83e7e7488673bb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:48.054386 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-004cb69d-021d-417a-84c0-8ef096d1687b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7e129cf70ab46418e83e7e7488673bb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:48.057752 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-004cb69d-021d-417a-84c0-8ef096d1687b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7e129cf70ab46418e83e7e7488673bb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:48.080847 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-004cb69d-021d-417a-84c0-8ef096d1687b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7e129cf70ab46418e83e7e7488673bb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:48.107777 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1097/2230] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:31:47 2022] DELETE /identity/v3/users/e7e129cf70ab46418e83e7e7488673bb => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:48.453005 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6e8f2f-f288-4495-aebb-7ffd0d0148e4 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:48.455069 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-af6e8f2f-f288-4495-aebb-7ffd0d0148e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:48.506109 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004cb69d-021d-417a-84c0-8ef096d1687b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:48.508817 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-004cb69d-021d-417a-84c0-8ef096d1687b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:48.560985 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6e8f2f-f288-4495-aebb-7ffd0d0148e4 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:48.698732 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a495351c-4b9f-4cdb-aa5b-3459a28d9e6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:48.700287 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a495351c-4b9f-4cdb-aa5b-3459a28d9e6e service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:48.700744 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a495351c-4b9f-4cdb-aa5b-3459a28d9e6e service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:48.701034 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a495351c-4b9f-4cdb-aa5b-3459a28d9e6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:48.704189 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a495351c-4b9f-4cdb-aa5b-3459a28d9e6e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:48.727243 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:48.901206 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a495351c-4b9f-4cdb-aa5b-3459a28d9e6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:48.903398 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004cb69d-021d-417a-84c0-8ef096d1687b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:48.907949 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a495351c-4b9f-4cdb-aa5b-3459a28d9e6e service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:48.922503 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6523ce1d-214c-4457-bbd2-ef3a6142952e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:48.924628 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6523ce1d-214c-4457-bbd2-ef3a6142952e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:48.925378 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6523ce1d-214c-4457-bbd2-ef3a6142952e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:48.926092 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6523ce1d-214c-4457-bbd2-ef3a6142952e admin admin] PATH_INFO: `/v3/projects/ae50dbc716e04b0daee1fd5c697dcea7` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:48.928837 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6523ce1d-214c-4457-bbd2-ef3a6142952e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae50dbc716e04b0daee1fd5c697dcea7)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:48.936598 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6523ce1d-214c-4457-bbd2-ef3a6142952e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:48.966087 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1134/2231] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:31:48 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:31:48.988752 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:49.050936 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6523ce1d-214c-4457-bbd2-ef3a6142952e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae50dbc716e04b0daee1fd5c697dcea7', 'request_id': 'req-6523ce1d-214c-4457-bbd2-ef3a6142952e'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:49.077464 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6523ce1d-214c-4457-bbd2-ef3a6142952e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae50dbc716e04b0daee1fd5c697dcea7', 'request_id': 'req-6523ce1d-214c-4457-bbd2-ef3a6142952e'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:49.099059 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6523ce1d-214c-4457-bbd2-ef3a6142952e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae50dbc716e04b0daee1fd5c697dcea7', 'request_id': 'req-6523ce1d-214c-4457-bbd2-ef3a6142952e'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:49.129200 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6523ce1d-214c-4457-bbd2-ef3a6142952e admin admin] The token cache is being invalidate because project ae50dbc716e04b0daee1fd5c697dcea7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:31:49.129919 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6523ce1d-214c-4457-bbd2-ef3a6142952e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:49.142956 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6523ce1d-214c-4457-bbd2-ef3a6142952e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:49.146634 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1098/2232] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:31:48 2022] DELETE /identity/v3/projects/ae50dbc716e04b0daee1fd5c697dcea7 => generated 0 bytes in 641 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:31:49.153158 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4179508b-1fbf-4c7d-a4cd-c8f57d051e34 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:49.153158 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4179508b-1fbf-4c7d-a4cd-c8f57d051e34 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:49.153899 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4179508b-1fbf-4c7d-a4cd-c8f57d051e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:49.248222 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-4179508b-1fbf-4c7d-a4cd-c8f57d051e34 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:49.250942 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4179508b-1fbf-4c7d-a4cd-c8f57d051e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:49.423977 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1135/2233] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:31:49.427065 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca64b0d4-75c0-40c7-8863-fb5315c87539 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:49.427550 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca64b0d4-75c0-40c7-8863-fb5315c87539 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:49.428667 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca64b0d4-75c0-40c7-8863-fb5315c87539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:49.469987 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-ca64b0d4-75c0-40c7-8863-fb5315c87539 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:49.471903 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ca64b0d4-75c0-40c7-8863-fb5315c87539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:49.492170 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1099/2234] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:31:49.499740 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4179508b-1fbf-4c7d-a4cd-c8f57d051e34 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:49.515645 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4179508b-1fbf-4c7d-a4cd-c8f57d051e34 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:49.526840 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a62c851-cddf-413a-a398-26887bb886bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:49.528484 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62c851-cddf-413a-a398-26887bb886bb admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:49.528804 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62c851-cddf-413a-a398-26887bb886bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:49.529273 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62c851-cddf-413a-a398-26887bb886bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:49.530640 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a62c851-cddf-413a-a398-26887bb886bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:31:49.531488 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a62c851-cddf-413a-a398-26887bb886bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:49.534179 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a62c851-cddf-413a-a398-26887bb886bb admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:49.557478 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1136/2235] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:31:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:31:49.566344 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca64b0d4-75c0-40c7-8863-fb5315c87539 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:49.591617 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca64b0d4-75c0-40c7-8863-fb5315c87539 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:49.598101 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb97c5-410a-43bc-9d4a-e196e123b62f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:49.599094 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb97c5-410a-43bc-9d4a-e196e123b62f admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:49.599442 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb97c5-410a-43bc-9d4a-e196e123b62f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:49.599775 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb97c5-410a-43bc-9d4a-e196e123b62f admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:49.600561 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedb97c5-410a-43bc-9d4a-e196e123b62f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:49.602794 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedb97c5-410a-43bc-9d4a-e196e123b62f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:49.668781 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1100/2236] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:31:49 2022] POST /identity/v3/projects => generated 381 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:31:49.670741 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a62c851-cddf-413a-a398-26887bb886bb admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:49.694106 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a62c851-cddf-413a-a398-26887bb886bb admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:49.888064 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d9e89d-a345-4d28-acae-67b10ac628a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:49.889301 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d9e89d-a345-4d28-acae-67b10ac628a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:49.890012 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d9e89d-a345-4d28-acae-67b10ac628a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:49.890643 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d9e89d-a345-4d28-acae-67b10ac628a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:49.891507 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d9e89d-a345-4d28-acae-67b10ac628a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:49.894909 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d9e89d-a345-4d28-acae-67b10ac628a1 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:49.929035 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1137/2237] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:31:49 2022] POST /identity/v3/users => generated 291 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:31:49.932785 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb97c5-410a-43bc-9d4a-e196e123b62f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:49.954697 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb97c5-410a-43bc-9d4a-e196e123b62f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:49.971503 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93901eaa-2eb9-4c8e-be4c-76ab202e301e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:49.971503 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93901eaa-2eb9-4c8e-be4c-76ab202e301e admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:49.971503 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93901eaa-2eb9-4c8e-be4c-76ab202e301e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:49.971503 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93901eaa-2eb9-4c8e-be4c-76ab202e301e admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:49.972716 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93901eaa-2eb9-4c8e-be4c-76ab202e301e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:49.974866 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93901eaa-2eb9-4c8e-be4c-76ab202e301e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:49.994236 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1101/2238] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:31:49 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:31:49.997177 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d9e89d-a345-4d28-acae-67b10ac628a1 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:50.014843 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d9e89d-a345-4d28-acae-67b10ac628a1 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:50.021379 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc627bf-0728-435c-aa2b-14f8e5cbaa0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:50.022320 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc627bf-0728-435c-aa2b-14f8e5cbaa0a admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:50.022834 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc627bf-0728-435c-aa2b-14f8e5cbaa0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:50.023356 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc627bf-0728-435c-aa2b-14f8e5cbaa0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:50.024411 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc627bf-0728-435c-aa2b-14f8e5cbaa0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:50.026921 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc627bf-0728-435c-aa2b-14f8e5cbaa0a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:50.041134 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1138/2239] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:31:49 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:31:50.048042 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93901eaa-2eb9-4c8e-be4c-76ab202e301e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:50.072953 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93901eaa-2eb9-4c8e-be4c-76ab202e301e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:50.082199 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d1af89-1058-4b87-baa9-fdb11046e7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:50.083801 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1af89-1058-4b87-baa9-fdb11046e7c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:50.084568 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1af89-1058-4b87-baa9-fdb11046e7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:50.085140 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1af89-1058-4b87-baa9-fdb11046e7c3 admin admin] PATH_INFO: `/v3/projects/4b6c78c494104b12b343574c348e21b1/users/f4caf4b7c67a43ea86fe0219cf1cf862/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:50.118372 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d1af89-1058-4b87-baa9-fdb11046e7c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b6c78c494104b12b343574c348e21b1, user_id=f4caf4b7c67a43ea86fe0219cf1cf862, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:50.123691 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d1af89-1058-4b87-baa9-fdb11046e7c3 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:50.143229 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1102/2240] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:31:50 2022] PUT /identity/v3/projects/4b6c78c494104b12b343574c348e21b1/users/f4caf4b7c67a43ea86fe0219cf1cf862/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:31:50.158176 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7c1a2-1222-4728-8059-121bf31a1691 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:50.158680 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7c1a2-1222-4728-8059-121bf31a1691 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:50.159347 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7c1a2-1222-4728-8059-121bf31a1691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:50.195080 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:50.214873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-d1b7c1a2-1222-4728-8059-121bf31a1691 None None] MFA Rules not processed for user `f4caf4b7c67a43ea86fe0219cf1cf862`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:50.216172 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d1b7c1a2-1222-4728-8059-121bf31a1691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:50.353622 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1139/2241] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:31:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:31:50.373825 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d1af89-1058-4b87-baa9-fdb11046e7c3 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:50.376356 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a9d1af89-1058-4b87-baa9-fdb11046e7c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:50.567940 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d1af89-1058-4b87-baa9-fdb11046e7c3 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:50.576538 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd34e75-1381-4989-b75e-a40b16ba8c93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:50.577136 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd34e75-1381-4989-b75e-a40b16ba8c93 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:50.577659 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd34e75-1381-4989-b75e-a40b16ba8c93 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:50.578542 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd34e75-1381-4989-b75e-a40b16ba8c93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:50.600480 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd34e75-1381-4989-b75e-a40b16ba8c93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:50.604597 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd34e75-1381-4989-b75e-a40b16ba8c93 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:50.652857 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1103/2242] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:31:50 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:32:00.347583 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=27 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:32:00.633132 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=69,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:32:02.556835 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493bcc5-24c0-4551-a352-28b454e0ba69 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:32:02.557824 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493bcc5-24c0-4551-a352-28b454e0ba69 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:32:02.558002 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493bcc5-24c0-4551-a352-28b454e0ba69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:32:02.588093 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-9493bcc5-24c0-4551-a352-28b454e0ba69 None None] MFA Rules not processed for user `de09b91800314d06aeddafe3c12a77b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:32:02.591011 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-9493bcc5-24c0-4551-a352-28b454e0ba69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:32:02.686209 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1140/2243] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:32:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:32:12.678241 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=25 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:32:22.058938 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd34e75-1381-4989-b75e-a40b16ba8c93 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:32:22.061416 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2cd34e75-1381-4989-b75e-a40b16ba8c93 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:32:22.205624 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd34e75-1381-4989-b75e-a40b16ba8c93 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:32:22.214879 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e821ad-d44d-4c27-8646-d088db2890b4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef1d49b36cdc4665b82487be97e9dcd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:32:22.216061 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e821ad-d44d-4c27-8646-d088db2890b4 service placement] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:32:22.216598 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e821ad-d44d-4c27-8646-d088db2890b4 service placement] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:32:22.217091 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e821ad-d44d-4c27-8646-d088db2890b4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:32:22.220727 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-18e821ad-d44d-4c27-8646-d088db2890b4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:32:22.245315 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e821ad-d44d-4c27-8646-d088db2890b4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:32:22.248777 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e821ad-d44d-4c27-8646-d088db2890b4 service placement] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:32:22.273123 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1104/2244] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:32:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:32:25.094373 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9493bcc5-24c0-4551-a352-28b454e0ba69 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:32:25.265829 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9493bcc5-24c0-4551-a352-28b454e0ba69 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:32:25.274827 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375daf45-04b6-417d-b7ea-9ee994ee9f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:32:25.276390 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375daf45-04b6-417d-b7ea-9ee994ee9f1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:32:25.276901 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375daf45-04b6-417d-b7ea-9ee994ee9f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:32:25.278201 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375daf45-04b6-417d-b7ea-9ee994ee9f1c admin admin] PATH_INFO: `/v3/users/f4caf4b7c67a43ea86fe0219cf1cf862` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:32:25.280931 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375daf45-04b6-417d-b7ea-9ee994ee9f1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4caf4b7c67a43ea86fe0219cf1cf862)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:32:25.284423 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375daf45-04b6-417d-b7ea-9ee994ee9f1c admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:32:25.400754 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-375daf45-04b6-417d-b7ea-9ee994ee9f1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4caf4b7c67a43ea86fe0219cf1cf862'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:32:25.434886 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-375daf45-04b6-417d-b7ea-9ee994ee9f1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4caf4b7c67a43ea86fe0219cf1cf862'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:32:25.435984 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-375daf45-04b6-417d-b7ea-9ee994ee9f1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4caf4b7c67a43ea86fe0219cf1cf862'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:32:25.436972 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-375daf45-04b6-417d-b7ea-9ee994ee9f1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4caf4b7c67a43ea86fe0219cf1cf862'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:32:25.467346 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-375daf45-04b6-417d-b7ea-9ee994ee9f1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4caf4b7c67a43ea86fe0219cf1cf862'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:32:25.505833 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1141/2245] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:32:25 2022] DELETE /identity/v3/users/f4caf4b7c67a43ea86fe0219cf1cf862 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:32:25.507966 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7e8c4-6010-428a-ad1d-032cb7a6e6b6 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:32:25.508774 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7e8c4-6010-428a-ad1d-032cb7a6e6b6 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:32:25.508774 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7e8c4-6010-428a-ad1d-032cb7a6e6b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:32:25.547212 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-53b7e8c4-6010-428a-ad1d-032cb7a6e6b6 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:32:25.550196 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-53b7e8c4-6010-428a-ad1d-032cb7a6e6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:32:25.752717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1105/2246] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:32:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:32:25.769502 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375daf45-04b6-417d-b7ea-9ee994ee9f1c admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:32:25.771795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-375daf45-04b6-417d-b7ea-9ee994ee9f1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:32:25.896915 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375daf45-04b6-417d-b7ea-9ee994ee9f1c admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:32:25.905026 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda2bed6-9d5a-4c8f-b0f8-ce2db050da53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:32:25.906184 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda2bed6-9d5a-4c8f-b0f8-ce2db050da53 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:32:25.906658 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda2bed6-9d5a-4c8f-b0f8-ce2db050da53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:32:25.907123 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda2bed6-9d5a-4c8f-b0f8-ce2db050da53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:32:25.929250 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda2bed6-9d5a-4c8f-b0f8-ce2db050da53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:32:25.934721 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda2bed6-9d5a-4c8f-b0f8-ce2db050da53 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:32:25.953240 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1142/2247] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:32:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:32:26.614976 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b7e8c4-6010-428a-ad1d-032cb7a6e6b6 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:32:26.618602 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-53b7e8c4-6010-428a-ad1d-032cb7a6e6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:32:26.644918 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b7e8c4-6010-428a-ad1d-032cb7a6e6b6 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:32:26.657917 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a9b5c-ec7b-4d31-a1a5-734462f5ce35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:32:26.659914 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a9b5c-ec7b-4d31-a1a5-734462f5ce35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:32:26.660632 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a9b5c-ec7b-4d31-a1a5-734462f5ce35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:32:26.661466 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a9b5c-ec7b-4d31-a1a5-734462f5ce35 admin admin] PATH_INFO: `/v3/projects/4b6c78c494104b12b343574c348e21b1` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:32:26.663910 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8a9b5c-ec7b-4d31-a1a5-734462f5ce35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b6c78c494104b12b343574c348e21b1)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:32:26.666752 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8a9b5c-ec7b-4d31-a1a5-734462f5ce35 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:32:26.792615 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ad8a9b5c-ec7b-4d31-a1a5-734462f5ce35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b6c78c494104b12b343574c348e21b1', 'request_id': 'req-ad8a9b5c-ec7b-4d31-a1a5-734462f5ce35'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:32:26.817504 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ad8a9b5c-ec7b-4d31-a1a5-734462f5ce35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b6c78c494104b12b343574c348e21b1', 'request_id': 'req-ad8a9b5c-ec7b-4d31-a1a5-734462f5ce35'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:32:26.832265 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ad8a9b5c-ec7b-4d31-a1a5-734462f5ce35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b6c78c494104b12b343574c348e21b1', 'request_id': 'req-ad8a9b5c-ec7b-4d31-a1a5-734462f5ce35'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:32:26.858054 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ad8a9b5c-ec7b-4d31-a1a5-734462f5ce35 admin admin] The token cache is being invalidate because project 4b6c78c494104b12b343574c348e21b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:32:26.858556 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ad8a9b5c-ec7b-4d31-a1a5-734462f5ce35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:32:26.859733 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ad8a9b5c-ec7b-4d31-a1a5-734462f5ce35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:32:26.862787 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1106/2248] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:32:26 2022] DELETE /identity/v3/projects/4b6c78c494104b12b343574c348e21b1 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:32:26.870856 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9d9cd-389c-4b10-9421-769e0b028104 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:32:26.871613 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9d9cd-389c-4b10-9421-769e0b028104 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:32:26.871910 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9d9cd-389c-4b10-9421-769e0b028104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:32:26.905748 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-55f9d9cd-389c-4b10-9421-769e0b028104 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:32:26.907713 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-55f9d9cd-389c-4b10-9421-769e0b028104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:32:26.936168 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:32:27.017554 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1143/2249] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:32:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:32:27.029378 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604c057-dbc1-4561-94f7-f805c172e42c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:32:27.029748 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604c057-dbc1-4561-94f7-f805c172e42c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:32:27.029984 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604c057-dbc1-4561-94f7-f805c172e42c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:32:27.056264 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-0604c057-dbc1-4561-94f7-f805c172e42c None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:32:27.058994 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0604c057-dbc1-4561-94f7-f805c172e42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:32:27.071244 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1107/2250] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:32:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:32:27.078540 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f9d9cd-389c-4b10-9421-769e0b028104 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:32:27.099352 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f9d9cd-389c-4b10-9421-769e0b028104 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:32:27.108514 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fb6224-08bc-484d-8319-54a2f94c5fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:32:27.109695 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb6224-08bc-484d-8319-54a2f94c5fc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:32:27.110233 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb6224-08bc-484d-8319-54a2f94c5fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:32:27.110737 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb6224-08bc-484d-8319-54a2f94c5fc2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:32:27.112149 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fb6224-08bc-484d-8319-54a2f94c5fc2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:32:27.112720 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fb6224-08bc-484d-8319-54a2f94c5fc2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:32:27.115895 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fb6224-08bc-484d-8319-54a2f94c5fc2 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:32:27.137307 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1144/2251] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:32:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:32:27.143886 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0604c057-dbc1-4561-94f7-f805c172e42c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:32:27.159737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0604c057-dbc1-4561-94f7-f805c172e42c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:32:27.166415 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4866cf-bdc8-4a8d-b245-8d368f61ccb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:32:27.167238 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4866cf-bdc8-4a8d-b245-8d368f61ccb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:32:27.167630 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4866cf-bdc8-4a8d-b245-8d368f61ccb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:32:27.168183 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4866cf-bdc8-4a8d-b245-8d368f61ccb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:32:27.169116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4866cf-bdc8-4a8d-b245-8d368f61ccb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:32:27.171340 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4866cf-bdc8-4a8d-b245-8d368f61ccb6 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:32:27.176428 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:32:27.215195 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1108/2252] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:32:27 2022] POST /identity/v3/projects => generated 373 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:32:27.220993 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fb6224-08bc-484d-8319-54a2f94c5fc2 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:32:27.239961 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fb6224-08bc-484d-8319-54a2f94c5fc2 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:32:27.322915 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1844aa7-49d7-48fd-9602-4c2d1af49c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:32:27.323985 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1844aa7-49d7-48fd-9602-4c2d1af49c37 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:32:27.324500 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1844aa7-49d7-48fd-9602-4c2d1af49c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:32:27.324974 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1844aa7-49d7-48fd-9602-4c2d1af49c37 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:32:27.326440 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1844aa7-49d7-48fd-9602-4c2d1af49c37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:32:27.330446 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1844aa7-49d7-48fd-9602-4c2d1af49c37 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:32:27.363860 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1145/2253] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:32:27 2022] POST /identity/v3/users => generated 287 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:32:27.369613 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4866cf-bdc8-4a8d-b245-8d368f61ccb6 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:32:27.389760 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4866cf-bdc8-4a8d-b245-8d368f61ccb6 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:32:27.399362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7360526-63b0-4a42-8e32-bb0500091d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:32:27.400454 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7360526-63b0-4a42-8e32-bb0500091d16 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:32:27.400934 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7360526-63b0-4a42-8e32-bb0500091d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:32:27.401430 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7360526-63b0-4a42-8e32-bb0500091d16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:32:27.402920 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7360526-63b0-4a42-8e32-bb0500091d16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:32:27.405773 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7360526-63b0-4a42-8e32-bb0500091d16 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:32:27.428474 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1109/2254] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:32:27 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:32:27.434315 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1844aa7-49d7-48fd-9602-4c2d1af49c37 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:32:27.449981 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1844aa7-49d7-48fd-9602-4c2d1af49c37 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:32:27.462964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc04108-323d-4997-8234-dc88ad4a52cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:32:27.464160 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc04108-323d-4997-8234-dc88ad4a52cd admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:32:27.464854 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc04108-323d-4997-8234-dc88ad4a52cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:32:27.465467 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc04108-323d-4997-8234-dc88ad4a52cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:32:27.466968 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc04108-323d-4997-8234-dc88ad4a52cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:32:27.470099 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc04108-323d-4997-8234-dc88ad4a52cd admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:32:27.497722 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1146/2255] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:32:27 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:32:27.500855 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7360526-63b0-4a42-8e32-bb0500091d16 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:32:27.516357 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7360526-63b0-4a42-8e32-bb0500091d16 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:32:27.531679 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf906050-6f49-4e75-84bf-e8f0222c4df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:32:27.532835 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf906050-6f49-4e75-84bf-e8f0222c4df8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:32:27.533130 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf906050-6f49-4e75-84bf-e8f0222c4df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:32:27.533419 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf906050-6f49-4e75-84bf-e8f0222c4df8 admin admin] PATH_INFO: `/v3/projects/b5569463610e44dc82d15553cbbcb02c/users/2bbfa51f793c4d2fb53ce386f4d33192/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:32:27.557104 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf906050-6f49-4e75-84bf-e8f0222c4df8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5569463610e44dc82d15553cbbcb02c, user_id=2bbfa51f793c4d2fb53ce386f4d33192, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:32:27.559998 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf906050-6f49-4e75-84bf-e8f0222c4df8 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:32:27.580794 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1110/2256] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:32:27 2022] PUT /identity/v3/projects/b5569463610e44dc82d15553cbbcb02c/users/2bbfa51f793c4d2fb53ce386f4d33192/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:32:27.591861 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac37cb-525a-4af9-bdb0-9db7ecd93eaf None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:32:27.592327 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac37cb-525a-4af9-bdb0-9db7ecd93eaf None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:32:27.592701 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac37cb-525a-4af9-bdb0-9db7ecd93eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:32:27.633387 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-b0ac37cb-525a-4af9-bdb0-9db7ecd93eaf None None] MFA Rules not processed for user `2bbfa51f793c4d2fb53ce386f4d33192`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:32:27.635314 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b0ac37cb-525a-4af9-bdb0-9db7ecd93eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:32:27.730758 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1147/2257] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:32:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:32:27.746687 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf906050-6f49-4e75-84bf-e8f0222c4df8 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:32:27.749395 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-bf906050-6f49-4e75-84bf-e8f0222c4df8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:32:27.855280 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf906050-6f49-4e75-84bf-e8f0222c4df8 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:32:27.859453 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22314df8-3d08-4600-8ad1-9dec55e30af4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:32:27.860571 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22314df8-3d08-4600-8ad1-9dec55e30af4 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:32:27.860922 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22314df8-3d08-4600-8ad1-9dec55e30af4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:32:27.861238 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22314df8-3d08-4600-8ad1-9dec55e30af4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:32:27.873821 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22314df8-3d08-4600-8ad1-9dec55e30af4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:32:27.875443 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22314df8-3d08-4600-8ad1-9dec55e30af4 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:32:27.890680 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1111/2258] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:32:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:32:37.726787 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:32:37.883193 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=47,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:32:38.092689 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e25eb1-877b-41eb-8971-113f7b456a36 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:32:38.093137 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e25eb1-877b-41eb-8971-113f7b456a36 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:32:38.093481 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e25eb1-877b-41eb-8971-113f7b456a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:32:38.122590 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-25e25eb1-877b-41eb-8971-113f7b456a36 None None] MFA Rules not processed for user `2bbfa51f793c4d2fb53ce386f4d33192`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:32:38.125283 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-25e25eb1-877b-41eb-8971-113f7b456a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:32:38.135379 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1148/2259] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:32:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:32:48.112832 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:32:51.484408 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22314df8-3d08-4600-8ad1-9dec55e30af4 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:32:51.487136 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-22314df8-3d08-4600-8ad1-9dec55e30af4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:32:51.600202 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22314df8-3d08-4600-8ad1-9dec55e30af4 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:32:51.608313 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f481c-f8bf-4c42-80c3-927491137674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:32:51.609266 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f481c-f8bf-4c42-80c3-927491137674 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:32:51.609602 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f481c-f8bf-4c42-80c3-927491137674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:32:51.609972 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f481c-f8bf-4c42-80c3-927491137674 admin admin] PATH_INFO: `/v3/users/a9e96baff8144ef18eb608bc94404719` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:32:51.611346 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162f481c-f8bf-4c42-80c3-927491137674 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9e96baff8144ef18eb608bc94404719)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:32:51.613541 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162f481c-f8bf-4c42-80c3-927491137674 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:32:51.694268 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-162f481c-f8bf-4c42-80c3-927491137674 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9e96baff8144ef18eb608bc94404719'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:32:51.719049 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-162f481c-f8bf-4c42-80c3-927491137674 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9e96baff8144ef18eb608bc94404719'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:32:51.719816 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-162f481c-f8bf-4c42-80c3-927491137674 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9e96baff8144ef18eb608bc94404719'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:32:51.720544 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-162f481c-f8bf-4c42-80c3-927491137674 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9e96baff8144ef18eb608bc94404719'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:32:51.746648 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-162f481c-f8bf-4c42-80c3-927491137674 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9e96baff8144ef18eb608bc94404719'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:32:51.771401 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1112/2260] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:32:51 2022] DELETE /identity/v3/users/a9e96baff8144ef18eb608bc94404719 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:32:51.777282 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc225538-09f1-43f3-9215-f1aa011ec99e None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:32:51.778082 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc225538-09f1-43f3-9215-f1aa011ec99e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:32:51.778082 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc225538-09f1-43f3-9215-f1aa011ec99e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:32:51.817078 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-dc225538-09f1-43f3-9215-f1aa011ec99e None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:32:51.819687 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-dc225538-09f1-43f3-9215-f1aa011ec99e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:32:51.984840 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1149/2261] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:32:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:32:51.996834 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f481c-f8bf-4c42-80c3-927491137674 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:32:51.999485 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-162f481c-f8bf-4c42-80c3-927491137674 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:32:52.113144 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f481c-f8bf-4c42-80c3-927491137674 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:32:52.121058 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac710c-62e3-419b-ae84-218d1c8e85e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:32:52.122340 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac710c-62e3-419b-ae84-218d1c8e85e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:32:52.122938 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac710c-62e3-419b-ae84-218d1c8e85e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:32:52.123515 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac710c-62e3-419b-ae84-218d1c8e85e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:32:52.151858 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ac710c-62e3-419b-ae84-218d1c8e85e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:32:52.153681 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ac710c-62e3-419b-ae84-218d1c8e85e2 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:32:52.173262 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1113/2262] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:32:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:32:52.710087 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc225538-09f1-43f3-9215-f1aa011ec99e None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:32:52.713188 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-dc225538-09f1-43f3-9215-f1aa011ec99e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:32:52.749560 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc225538-09f1-43f3-9215-f1aa011ec99e None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:32:52.758301 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1eb13e-bdd5-4180-86b0-df861b3aed85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:32:52.759761 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1eb13e-bdd5-4180-86b0-df861b3aed85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:32:52.760426 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1eb13e-bdd5-4180-86b0-df861b3aed85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:32:52.760730 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1eb13e-bdd5-4180-86b0-df861b3aed85 admin admin] PATH_INFO: `/v3/projects/bd4546e76587483cba926040be2b5832` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:32:52.763252 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1eb13e-bdd5-4180-86b0-df861b3aed85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd4546e76587483cba926040be2b5832)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:32:52.767133 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1eb13e-bdd5-4180-86b0-df861b3aed85 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:32:52.862236 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0d1eb13e-bdd5-4180-86b0-df861b3aed85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd4546e76587483cba926040be2b5832', 'request_id': 'req-0d1eb13e-bdd5-4180-86b0-df861b3aed85'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:32:52.871494 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0d1eb13e-bdd5-4180-86b0-df861b3aed85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd4546e76587483cba926040be2b5832', 'request_id': 'req-0d1eb13e-bdd5-4180-86b0-df861b3aed85'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:32:52.881482 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0d1eb13e-bdd5-4180-86b0-df861b3aed85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd4546e76587483cba926040be2b5832', 'request_id': 'req-0d1eb13e-bdd5-4180-86b0-df861b3aed85'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:32:52.894516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0d1eb13e-bdd5-4180-86b0-df861b3aed85 admin admin] The token cache is being invalidate because project bd4546e76587483cba926040be2b5832 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:32:52.894811 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0d1eb13e-bdd5-4180-86b0-df861b3aed85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:32:52.895801 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0d1eb13e-bdd5-4180-86b0-df861b3aed85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:32:52.897928 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1150/2263] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:32:52 2022] DELETE /identity/v3/projects/bd4546e76587483cba926040be2b5832 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:32:52.902828 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac710c-62e3-419b-ae84-218d1c8e85e2 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:32:52.915682 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac710c-62e3-419b-ae84-218d1c8e85e2 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:32:52.942321 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:32:52.992193 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78492d0a-5620-43b8-9dfc-cd91872d34fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:32:52.993542 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78492d0a-5620-43b8-9dfc-cd91872d34fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:32:52.993542 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78492d0a-5620-43b8-9dfc-cd91872d34fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:32:52.994049 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78492d0a-5620-43b8-9dfc-cd91872d34fc admin admin] PATH_INFO: `/v3/users/04ecdb2067df4962aef6a6a7ad8cbd61` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:32:52.995996 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78492d0a-5620-43b8-9dfc-cd91872d34fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=04ecdb2067df4962aef6a6a7ad8cbd61)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:32:52.998254 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78492d0a-5620-43b8-9dfc-cd91872d34fc admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:32:53.073446 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-78492d0a-5620-43b8-9dfc-cd91872d34fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04ecdb2067df4962aef6a6a7ad8cbd61'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:32:53.082154 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-78492d0a-5620-43b8-9dfc-cd91872d34fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04ecdb2067df4962aef6a6a7ad8cbd61'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:32:53.083249 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-78492d0a-5620-43b8-9dfc-cd91872d34fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04ecdb2067df4962aef6a6a7ad8cbd61'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:32:53.084152 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-78492d0a-5620-43b8-9dfc-cd91872d34fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04ecdb2067df4962aef6a6a7ad8cbd61'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:32:53.103644 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-78492d0a-5620-43b8-9dfc-cd91872d34fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04ecdb2067df4962aef6a6a7ad8cbd61'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:32:53.130224 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1114/2264] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:32:52 2022] DELETE /identity/v3/users/04ecdb2067df4962aef6a6a7ad8cbd61 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:32:53.854111 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1eb13e-bdd5-4180-86b0-df861b3aed85 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:32:53.856540 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-0d1eb13e-bdd5-4180-86b0-df861b3aed85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:32:53.893406 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1eb13e-bdd5-4180-86b0-df861b3aed85 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:32:54.122425 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7826e0-b929-42ca-8310-37a962c665c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:32:54.128957 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7826e0-b929-42ca-8310-37a962c665c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:32:54.130870 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7826e0-b929-42ca-8310-37a962c665c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:32:54.133378 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7826e0-b929-42ca-8310-37a962c665c4 admin admin] PATH_INFO: `/v3/projects/7c40fb6c011f47bdb8f89fdb56d88e69` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:32:54.140101 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7826e0-b929-42ca-8310-37a962c665c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c40fb6c011f47bdb8f89fdb56d88e69)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:32:54.148108 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7826e0-b929-42ca-8310-37a962c665c4 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:32:54.287333 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:32:54.346341 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-4c7826e0-b929-42ca-8310-37a962c665c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c40fb6c011f47bdb8f89fdb56d88e69', 'request_id': 'req-4c7826e0-b929-42ca-8310-37a962c665c4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:32:54.359945 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-4c7826e0-b929-42ca-8310-37a962c665c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c40fb6c011f47bdb8f89fdb56d88e69', 'request_id': 'req-4c7826e0-b929-42ca-8310-37a962c665c4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:32:54.375619 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-4c7826e0-b929-42ca-8310-37a962c665c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c40fb6c011f47bdb8f89fdb56d88e69', 'request_id': 'req-4c7826e0-b929-42ca-8310-37a962c665c4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:32:54.402229 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-4c7826e0-b929-42ca-8310-37a962c665c4 admin admin] The token cache is being invalidate because project 7c40fb6c011f47bdb8f89fdb56d88e69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:32:54.403192 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-4c7826e0-b929-42ca-8310-37a962c665c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:32:54.404735 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-4c7826e0-b929-42ca-8310-37a962c665c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:32:54.409011 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1151/2265] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:32:53 2022] DELETE /identity/v3/projects/7c40fb6c011f47bdb8f89fdb56d88e69 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:32:54.416793 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c08f7-76e9-4d42-b547-25b95990db31 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:32:54.417570 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c08f7-76e9-4d42-b547-25b95990db31 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:32:54.417570 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c08f7-76e9-4d42-b547-25b95990db31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:32:54.464654 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-542c08f7-76e9-4d42-b547-25b95990db31 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:32:54.467056 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-542c08f7-76e9-4d42-b547-25b95990db31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:32:54.632413 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1115/2266] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:32:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:32:54.648063 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289d8c5-bbfe-458f-97ce-ec2f45bfd405 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:32:54.648458 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289d8c5-bbfe-458f-97ce-ec2f45bfd405 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:32:54.648975 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289d8c5-bbfe-458f-97ce-ec2f45bfd405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:32:54.724678 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-d289d8c5-bbfe-458f-97ce-ec2f45bfd405 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:32:54.734608 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d289d8c5-bbfe-458f-97ce-ec2f45bfd405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:32:54.783907 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1152/2267] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:32:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:32:54.786451 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542c08f7-76e9-4d42-b547-25b95990db31 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:32:54.846604 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542c08f7-76e9-4d42-b547-25b95990db31 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:32:54.870062 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf476d6-6172-425a-b718-1f5cdc7f2f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:32:54.875375 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf476d6-6172-425a-b718-1f5cdc7f2f18 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:32:54.876842 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf476d6-6172-425a-b718-1f5cdc7f2f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:32:54.878627 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf476d6-6172-425a-b718-1f5cdc7f2f18 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:32:54.883396 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf476d6-6172-425a-b718-1f5cdc7f2f18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:32:54.886726 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf476d6-6172-425a-b718-1f5cdc7f2f18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:32:54.896272 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf476d6-6172-425a-b718-1f5cdc7f2f18 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:32:54.951861 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1116/2268] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:32:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:32:54.954230 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d289d8c5-bbfe-458f-97ce-ec2f45bfd405 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:32:54.981189 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d289d8c5-bbfe-458f-97ce-ec2f45bfd405 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:32:55.000456 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f268d959-84f4-4b51-90b8-270538d8f8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:32:55.002232 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268d959-84f4-4b51-90b8-270538d8f8f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:32:55.003452 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268d959-84f4-4b51-90b8-270538d8f8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:32:55.004178 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268d959-84f4-4b51-90b8-270538d8f8f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:32:55.005813 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f268d959-84f4-4b51-90b8-270538d8f8f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:32:55.017551 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f268d959-84f4-4b51-90b8-270538d8f8f2 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:32:55.112077 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1153/2269] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:32:54 2022] POST /identity/v3/projects => generated 367 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:32:55.118719 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf476d6-6172-425a-b718-1f5cdc7f2f18 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:32:55.138409 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf476d6-6172-425a-b718-1f5cdc7f2f18 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:32:55.204250 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:32:55.285199 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a7390f-c341-4c75-a6d9-f8af0fc45313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:32:55.286535 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7390f-c341-4c75-a6d9-f8af0fc45313 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:32:55.287084 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7390f-c341-4c75-a6d9-f8af0fc45313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:32:55.287585 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7390f-c341-4c75-a6d9-f8af0fc45313 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:32:55.288906 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a7390f-c341-4c75-a6d9-f8af0fc45313 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:32:55.291733 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a7390f-c341-4c75-a6d9-f8af0fc45313 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:32:55.329494 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1117/2270] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:32:55 2022] POST /identity/v3/users => generated 284 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:32:55.331204 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f268d959-84f4-4b51-90b8-270538d8f8f2 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:32:55.349713 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f268d959-84f4-4b51-90b8-270538d8f8f2 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:32:55.359253 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49224bf-dadf-4582-b034-e040a0f25172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:32:55.360614 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49224bf-dadf-4582-b034-e040a0f25172 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:32:55.361537 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49224bf-dadf-4582-b034-e040a0f25172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:32:55.362364 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49224bf-dadf-4582-b034-e040a0f25172 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:32:55.364024 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49224bf-dadf-4582-b034-e040a0f25172 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:32:55.367324 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49224bf-dadf-4582-b034-e040a0f25172 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:32:55.400536 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1154/2271] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:32:55 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:32:55.406394 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a7390f-c341-4c75-a6d9-f8af0fc45313 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:32:55.425463 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a7390f-c341-4c75-a6d9-f8af0fc45313 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:32:55.443895 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1381d4b8-f4bf-4702-9edb-9044e7251290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:32:55.444917 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381d4b8-f4bf-4702-9edb-9044e7251290 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:32:55.444917 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381d4b8-f4bf-4702-9edb-9044e7251290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:32:55.447433 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381d4b8-f4bf-4702-9edb-9044e7251290 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:32:55.447433 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1381d4b8-f4bf-4702-9edb-9044e7251290 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:32:55.448886 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1381d4b8-f4bf-4702-9edb-9044e7251290 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:32:55.513168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1118/2272] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:32:55 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:32:55.516145 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49224bf-dadf-4582-b034-e040a0f25172 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:32:55.530027 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49224bf-dadf-4582-b034-e040a0f25172 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:32:55.550998 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341367e9-e7ba-4aa3-a8e6-13cad260b9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:32:55.552989 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341367e9-e7ba-4aa3-a8e6-13cad260b9e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:32:55.553813 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341367e9-e7ba-4aa3-a8e6-13cad260b9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:32:55.554534 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341367e9-e7ba-4aa3-a8e6-13cad260b9e0 admin admin] PATH_INFO: `/v3/projects/20e97a1072ce4430954c478b5b423571/users/d6da4313d6054cdebb939f87b334ceca/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:32:55.588886 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341367e9-e7ba-4aa3-a8e6-13cad260b9e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20e97a1072ce4430954c478b5b423571, user_id=d6da4313d6054cdebb939f87b334ceca, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:32:55.592743 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341367e9-e7ba-4aa3-a8e6-13cad260b9e0 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:32:55.629326 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1155/2273] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:32:55 2022] PUT /identity/v3/projects/20e97a1072ce4430954c478b5b423571/users/d6da4313d6054cdebb939f87b334ceca/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:32:55.647993 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7137b4e-cf4e-495a-a87e-2e6c6e2ebc0a None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:32:55.648547 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7137b4e-cf4e-495a-a87e-2e6c6e2ebc0a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:32:55.649241 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7137b4e-cf4e-495a-a87e-2e6c6e2ebc0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:32:55.736110 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f7137b4e-cf4e-495a-a87e-2e6c6e2ebc0a None None] MFA Rules not processed for user `d6da4313d6054cdebb939f87b334ceca`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:32:55.738539 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f7137b4e-cf4e-495a-a87e-2e6c6e2ebc0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:32:55.913683 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1119/2274] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:32:55 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:32:55.920285 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341367e9-e7ba-4aa3-a8e6-13cad260b9e0 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:32:56.061281 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341367e9-e7ba-4aa3-a8e6-13cad260b9e0 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:32:56.066992 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d1f74c-b5cc-4201-9e9d-aea700dd20d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:32:56.067899 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1f74c-b5cc-4201-9e9d-aea700dd20d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:32:56.068330 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1f74c-b5cc-4201-9e9d-aea700dd20d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:32:56.068620 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1f74c-b5cc-4201-9e9d-aea700dd20d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:32:56.069544 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d1f74c-b5cc-4201-9e9d-aea700dd20d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:32:56.071998 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d1f74c-b5cc-4201-9e9d-aea700dd20d9 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:32:56.130317 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1156/2275] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:32:55 2022] POST /identity/v3/projects => generated 367 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:32:56.132966 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7137b4e-cf4e-495a-a87e-2e6c6e2ebc0a None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:32:56.177382 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7137b4e-cf4e-495a-a87e-2e6c6e2ebc0a None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:32:56.281117 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09b6578-b1c1-4839-8ff4-8ba73681360e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:32:56.282369 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b6578-b1c1-4839-8ff4-8ba73681360e admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:32:56.282901 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b6578-b1c1-4839-8ff4-8ba73681360e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:32:56.283483 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b6578-b1c1-4839-8ff4-8ba73681360e admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:32:56.284501 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09b6578-b1c1-4839-8ff4-8ba73681360e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:32:56.287643 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09b6578-b1c1-4839-8ff4-8ba73681360e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:32:56.321590 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1120/2276] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:32:56 2022] POST /identity/v3/users => generated 290 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:32:56.327318 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d1f74c-b5cc-4201-9e9d-aea700dd20d9 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:32:56.344565 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d1f74c-b5cc-4201-9e9d-aea700dd20d9 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:32:56.351086 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6439a9fb-ec46-4f47-bb4d-d8aac9ffb177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:32:56.351766 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439a9fb-ec46-4f47-bb4d-d8aac9ffb177 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:32:56.352085 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439a9fb-ec46-4f47-bb4d-d8aac9ffb177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:32:56.352598 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439a9fb-ec46-4f47-bb4d-d8aac9ffb177 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:32:56.353764 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6439a9fb-ec46-4f47-bb4d-d8aac9ffb177 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:32:56.356199 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6439a9fb-ec46-4f47-bb4d-d8aac9ffb177 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:32:56.372050 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1157/2277] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:32:56 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:32:56.379417 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09b6578-b1c1-4839-8ff4-8ba73681360e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:32:56.403696 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09b6578-b1c1-4839-8ff4-8ba73681360e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:32:56.429047 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ccdad-ebf2-4b8e-b730-7fab08552794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:32:56.431144 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ccdad-ebf2-4b8e-b730-7fab08552794 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:32:56.431661 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ccdad-ebf2-4b8e-b730-7fab08552794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:32:56.432233 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ccdad-ebf2-4b8e-b730-7fab08552794 admin admin] PATH_INFO: `/v3/projects/6b23803655564262836432f079631192/users/e9f8d33deb5944839a4bb609329998bd/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:32:56.463539 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192ccdad-ebf2-4b8e-b730-7fab08552794 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b23803655564262836432f079631192, user_id=e9f8d33deb5944839a4bb609329998bd, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:32:56.467867 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192ccdad-ebf2-4b8e-b730-7fab08552794 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:32:56.493312 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1121/2278] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:32:56 2022] PUT /identity/v3/projects/6b23803655564262836432f079631192/users/e9f8d33deb5944839a4bb609329998bd/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:32:56.499951 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6439a9fb-ec46-4f47-bb4d-d8aac9ffb177 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:32:56.516505 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6439a9fb-ec46-4f47-bb4d-d8aac9ffb177 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:32:56.567616 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:32:56.648292 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4b8438-42e2-4f8d-9d54-3ad394420a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:32:56.649447 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b8438-42e2-4f8d-9d54-3ad394420a2a admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:32:56.650045 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b8438-42e2-4f8d-9d54-3ad394420a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:32:56.650534 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b8438-42e2-4f8d-9d54-3ad394420a2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:32:56.651857 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4b8438-42e2-4f8d-9d54-3ad394420a2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:32:56.655484 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4b8438-42e2-4f8d-9d54-3ad394420a2a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:32:56.678530 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1158/2279] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:32:56 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:32:56.684530 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ccdad-ebf2-4b8e-b730-7fab08552794 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:32:56.695898 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:32:56.708913 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ccdad-ebf2-4b8e-b730-7fab08552794 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:32:56.717452 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55515b39-5315-4792-86ff-203082d36421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:32:56.718731 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55515b39-5315-4792-86ff-203082d36421 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:32:56.719364 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55515b39-5315-4792-86ff-203082d36421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:32:56.719951 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55515b39-5315-4792-86ff-203082d36421 admin admin] PATH_INFO: `/v3/projects/6b23803655564262836432f079631192/users/e9f8d33deb5944839a4bb609329998bd/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:32:56.724532 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55515b39-5315-4792-86ff-203082d36421 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b23803655564262836432f079631192, user_id=e9f8d33deb5944839a4bb609329998bd, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:32:56.730745 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55515b39-5315-4792-86ff-203082d36421 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:32:56.767671 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1122/2280] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:32:56 2022] PUT /identity/v3/projects/6b23803655564262836432f079631192/users/e9f8d33deb5944839a4bb609329998bd/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:32:56.785859 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5142c-04f6-4f61-b1eb-acbec50ae27c None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:32:56.786358 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5142c-04f6-4f61-b1eb-acbec50ae27c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:32:56.786844 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5142c-04f6-4f61-b1eb-acbec50ae27c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:32:56.880145 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-4bb5142c-04f6-4f61-b1eb-acbec50ae27c None None] MFA Rules not processed for user `e9f8d33deb5944839a4bb609329998bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:32:56.882383 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4bb5142c-04f6-4f61-b1eb-acbec50ae27c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:32:57.084763 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1159/2281] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:32:56 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:32:57.095206 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55515b39-5315-4792-86ff-203082d36421 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:32:57.097628 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-55515b39-5315-4792-86ff-203082d36421 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:32:57.264829 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55515b39-5315-4792-86ff-203082d36421 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:32:57.278520 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7508c63-2080-4162-b349-4119bc3c3ff1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:32:57.279531 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7508c63-2080-4162-b349-4119bc3c3ff1 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:32:57.280553 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7508c63-2080-4162-b349-4119bc3c3ff1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:32:57.281084 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7508c63-2080-4162-b349-4119bc3c3ff1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:32:57.295509 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7508c63-2080-4162-b349-4119bc3c3ff1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:32:57.298145 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7508c63-2080-4162-b349-4119bc3c3ff1 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:32:57.321736 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1123/2282] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:32:57 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:32:57.478135 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb5142c-04f6-4f61-b1eb-acbec50ae27c None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:32:57.482030 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4bb5142c-04f6-4f61-b1eb-acbec50ae27c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:32:57.518665 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb5142c-04f6-4f61-b1eb-acbec50ae27c None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:32:57.533811 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795e7e4d-b246-4c2c-aa2b-b2ae1445d5e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:32:57.535785 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e7e4d-b246-4c2c-aa2b-b2ae1445d5e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:32:57.536883 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e7e4d-b246-4c2c-aa2b-b2ae1445d5e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:32:57.537924 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e7e4d-b246-4c2c-aa2b-b2ae1445d5e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:32:57.721962 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795e7e4d-b246-4c2c-aa2b-b2ae1445d5e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:32:57.725806 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795e7e4d-b246-4c2c-aa2b-b2ae1445d5e0 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:32:57.743899 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1160/2283] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:32:57 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:33:02.268070 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7508c63-2080-4162-b349-4119bc3c3ff1 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:02.472878 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7508c63-2080-4162-b349-4119bc3c3ff1 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:02.481222 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4eec78-c52e-4286-a09d-b55b959711c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:02.482998 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4eec78-c52e-4286-a09d-b55b959711c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:02.483604 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4eec78-c52e-4286-a09d-b55b959711c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:02.484233 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4eec78-c52e-4286-a09d-b55b959711c4 admin admin] PATH_INFO: `/v3/users/d6da4313d6054cdebb939f87b334ceca` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:02.487816 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4eec78-c52e-4286-a09d-b55b959711c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6da4313d6054cdebb939f87b334ceca)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:02.491313 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4eec78-c52e-4286-a09d-b55b959711c4 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:02.596027 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-1f4eec78-c52e-4286-a09d-b55b959711c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6da4313d6054cdebb939f87b334ceca'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:02.612568 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-1f4eec78-c52e-4286-a09d-b55b959711c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6da4313d6054cdebb939f87b334ceca'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:02.612871 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-1f4eec78-c52e-4286-a09d-b55b959711c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6da4313d6054cdebb939f87b334ceca'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:02.614014 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-1f4eec78-c52e-4286-a09d-b55b959711c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6da4313d6054cdebb939f87b334ceca'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:02.639812 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-1f4eec78-c52e-4286-a09d-b55b959711c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6da4313d6054cdebb939f87b334ceca'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:02.664038 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1124/2284] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:33:02 2022] DELETE /identity/v3/users/d6da4313d6054cdebb939f87b334ceca => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:33:02.674112 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8d4dc-25f9-46ba-ac47-259b83a729c8 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:02.674840 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8d4dc-25f9-46ba-ac47-259b83a729c8 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:02.674840 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8d4dc-25f9-46ba-ac47-259b83a729c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:02.696655 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-67c8d4dc-25f9-46ba-ac47-259b83a729c8 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:02.698008 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-67c8d4dc-25f9-46ba-ac47-259b83a729c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:02.764329 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:02.793849 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1161/2285] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:33:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:33:02.808545 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4eec78-c52e-4286-a09d-b55b959711c4 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:02.810717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1f4eec78-c52e-4286-a09d-b55b959711c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:02.833924 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:02.903034 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4eec78-c52e-4286-a09d-b55b959711c4 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:02.907914 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad8ce4-5086-4393-a060-b0463af4195e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:02.908707 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad8ce4-5086-4393-a060-b0463af4195e service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:02.909185 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad8ce4-5086-4393-a060-b0463af4195e service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:02.909759 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad8ce4-5086-4393-a060-b0463af4195e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:02.929530 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ad8ce4-5086-4393-a060-b0463af4195e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:02.932333 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ad8ce4-5086-4393-a060-b0463af4195e service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:02.947022 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1125/2286] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:33:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:33:03.892632 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8d4dc-25f9-46ba-ac47-259b83a729c8 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:03.896619 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-67c8d4dc-25f9-46ba-ac47-259b83a729c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:04.002078 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8d4dc-25f9-46ba-ac47-259b83a729c8 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:04.021800 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f620ff-5543-48e5-86fa-e69d4b7ae35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:04.027811 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f620ff-5543-48e5-86fa-e69d4b7ae35d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:04.028380 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f620ff-5543-48e5-86fa-e69d4b7ae35d admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:04.028907 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f620ff-5543-48e5-86fa-e69d4b7ae35d admin admin] PATH_INFO: `/v3/projects/20e97a1072ce4430954c478b5b423571` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:04.033054 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f620ff-5543-48e5-86fa-e69d4b7ae35d admin admin] RBAC: Authorizing `identity:delete_project(project_id=20e97a1072ce4430954c478b5b423571)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:04.038120 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f620ff-5543-48e5-86fa-e69d4b7ae35d admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:04.155669 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-85f620ff-5543-48e5-86fa-e69d4b7ae35d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20e97a1072ce4430954c478b5b423571', 'request_id': 'req-85f620ff-5543-48e5-86fa-e69d4b7ae35d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:04.167509 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-85f620ff-5543-48e5-86fa-e69d4b7ae35d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20e97a1072ce4430954c478b5b423571', 'request_id': 'req-85f620ff-5543-48e5-86fa-e69d4b7ae35d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:04.184417 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-85f620ff-5543-48e5-86fa-e69d4b7ae35d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20e97a1072ce4430954c478b5b423571', 'request_id': 'req-85f620ff-5543-48e5-86fa-e69d4b7ae35d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:04.223948 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-85f620ff-5543-48e5-86fa-e69d4b7ae35d admin admin] The token cache is being invalidate because project 20e97a1072ce4430954c478b5b423571 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:33:04.226174 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-85f620ff-5543-48e5-86fa-e69d4b7ae35d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:04.228486 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-85f620ff-5543-48e5-86fa-e69d4b7ae35d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:04.240694 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1162/2287] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:33:03 2022] DELETE /identity/v3/projects/20e97a1072ce4430954c478b5b423571 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:04.247132 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad8ce4-5086-4393-a060-b0463af4195e service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:04.276517 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad8ce4-5086-4393-a060-b0463af4195e service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:04.614275 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b8f763-483f-48ea-a9e6-b6d0b6e6be76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:04.614275 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b8f763-483f-48ea-a9e6-b6d0b6e6be76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:04.614275 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b8f763-483f-48ea-a9e6-b6d0b6e6be76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:04.615162 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b8f763-483f-48ea-a9e6-b6d0b6e6be76 admin admin] PATH_INFO: `/v3/users/e9f8d33deb5944839a4bb609329998bd` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:04.616802 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b8f763-483f-48ea-a9e6-b6d0b6e6be76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9f8d33deb5944839a4bb609329998bd)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:04.621215 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b8f763-483f-48ea-a9e6-b6d0b6e6be76 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:04.746196 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b6b8f763-483f-48ea-a9e6-b6d0b6e6be76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9f8d33deb5944839a4bb609329998bd'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:04.765904 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b6b8f763-483f-48ea-a9e6-b6d0b6e6be76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9f8d33deb5944839a4bb609329998bd'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:04.771540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b6b8f763-483f-48ea-a9e6-b6d0b6e6be76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9f8d33deb5944839a4bb609329998bd'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:04.772730 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b6b8f763-483f-48ea-a9e6-b6d0b6e6be76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9f8d33deb5944839a4bb609329998bd'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:04.806013 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b6b8f763-483f-48ea-a9e6-b6d0b6e6be76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9f8d33deb5944839a4bb609329998bd'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:04.853421 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1126/2288] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:33:04 2022] DELETE /identity/v3/users/e9f8d33deb5944839a4bb609329998bd => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:05.538672 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f620ff-5543-48e5-86fa-e69d4b7ae35d admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:05.540472 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-85f620ff-5543-48e5-86fa-e69d4b7ae35d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:05.562037 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f620ff-5543-48e5-86fa-e69d4b7ae35d admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:05.669245 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442d82f9-3f8b-4881-b970-5ab491b66b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:05.670096 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442d82f9-3f8b-4881-b970-5ab491b66b12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:05.670382 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442d82f9-3f8b-4881-b970-5ab491b66b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:05.670662 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442d82f9-3f8b-4881-b970-5ab491b66b12 admin admin] PATH_INFO: `/v3/projects/6b23803655564262836432f079631192` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:05.672123 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442d82f9-3f8b-4881-b970-5ab491b66b12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b23803655564262836432f079631192)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:05.674154 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442d82f9-3f8b-4881-b970-5ab491b66b12 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:05.759425 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-442d82f9-3f8b-4881-b970-5ab491b66b12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b23803655564262836432f079631192', 'request_id': 'req-442d82f9-3f8b-4881-b970-5ab491b66b12'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:05.767784 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-442d82f9-3f8b-4881-b970-5ab491b66b12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b23803655564262836432f079631192', 'request_id': 'req-442d82f9-3f8b-4881-b970-5ab491b66b12'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:05.780061 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-442d82f9-3f8b-4881-b970-5ab491b66b12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b23803655564262836432f079631192', 'request_id': 'req-442d82f9-3f8b-4881-b970-5ab491b66b12'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:05.788982 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:05.796090 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-442d82f9-3f8b-4881-b970-5ab491b66b12 admin admin] The token cache is being invalidate because project 6b23803655564262836432f079631192 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:33:05.796358 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-442d82f9-3f8b-4881-b970-5ab491b66b12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:05.797185 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-442d82f9-3f8b-4881-b970-5ab491b66b12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:05.799445 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1163/2289] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:33:05 2022] DELETE /identity/v3/projects/6b23803655564262836432f079631192 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:05.805944 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f0dd6-47b0-41ce-b30e-50b59d1325f0 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:05.806976 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f0dd6-47b0-41ce-b30e-50b59d1325f0 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:05.807358 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f0dd6-47b0-41ce-b30e-50b59d1325f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:05.834688 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-764f0dd6-47b0-41ce-b30e-50b59d1325f0 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:05.835995 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-764f0dd6-47b0-41ce-b30e-50b59d1325f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:05.897897 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:05.961115 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1127/2290] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:33:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:33:05.973494 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26bcbac-a618-4fb1-b89b-94112c458872 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:05.973920 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26bcbac-a618-4fb1-b89b-94112c458872 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:05.974348 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26bcbac-a618-4fb1-b89b-94112c458872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:06.009857 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-a26bcbac-a618-4fb1-b89b-94112c458872 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:06.012055 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a26bcbac-a618-4fb1-b89b-94112c458872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:06.026731 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1164/2291] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:33:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:33:06.032117 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764f0dd6-47b0-41ce-b30e-50b59d1325f0 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:06.045318 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764f0dd6-47b0-41ce-b30e-50b59d1325f0 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:06.051016 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522df058-019b-4a82-9f8d-1f8adb8777fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:06.051720 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522df058-019b-4a82-9f8d-1f8adb8777fe admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:06.052013 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522df058-019b-4a82-9f8d-1f8adb8777fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:06.052293 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522df058-019b-4a82-9f8d-1f8adb8777fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:06.053051 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522df058-019b-4a82-9f8d-1f8adb8777fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:33:06.053367 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522df058-019b-4a82-9f8d-1f8adb8777fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:06.055138 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522df058-019b-4a82-9f8d-1f8adb8777fe admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:06.073881 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1128/2292] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:33:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:33:06.081273 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26bcbac-a618-4fb1-b89b-94112c458872 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:06.104456 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26bcbac-a618-4fb1-b89b-94112c458872 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:06.110809 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1c6aa-6af7-4e12-bdde-0e4f0fd6feab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:06.111614 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1c6aa-6af7-4e12-bdde-0e4f0fd6feab admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:06.112997 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1c6aa-6af7-4e12-bdde-0e4f0fd6feab admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:06.112997 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1c6aa-6af7-4e12-bdde-0e4f0fd6feab admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:06.112997 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f1c6aa-6af7-4e12-bdde-0e4f0fd6feab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:06.115381 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f1c6aa-6af7-4e12-bdde-0e4f0fd6feab admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:06.153057 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1165/2293] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:33:06 2022] POST /identity/v3/projects => generated 407 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:33:06.161479 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522df058-019b-4a82-9f8d-1f8adb8777fe admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:06.176262 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522df058-019b-4a82-9f8d-1f8adb8777fe admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:06.273268 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5838d68c-4109-4d06-87dd-42ca2322bf0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:06.275066 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5838d68c-4109-4d06-87dd-42ca2322bf0c admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:06.275580 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5838d68c-4109-4d06-87dd-42ca2322bf0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:06.276095 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5838d68c-4109-4d06-87dd-42ca2322bf0c admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:06.277444 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5838d68c-4109-4d06-87dd-42ca2322bf0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:06.281753 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5838d68c-4109-4d06-87dd-42ca2322bf0c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:06.317680 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1129/2294] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:33:06 2022] POST /identity/v3/users => generated 304 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:33:06.322884 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1c6aa-6af7-4e12-bdde-0e4f0fd6feab admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:06.336833 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1c6aa-6af7-4e12-bdde-0e4f0fd6feab admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:06.343936 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2578a5-1f3c-4bf4-854a-7117df64e74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:06.344795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2578a5-1f3c-4bf4-854a-7117df64e74b admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:06.345175 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2578a5-1f3c-4bf4-854a-7117df64e74b admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:06.345545 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2578a5-1f3c-4bf4-854a-7117df64e74b admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:06.346475 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2578a5-1f3c-4bf4-854a-7117df64e74b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:06.348284 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2578a5-1f3c-4bf4-854a-7117df64e74b admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:06.363413 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1166/2295] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:33:06 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:33:06.368823 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5838d68c-4109-4d06-87dd-42ca2322bf0c admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:06.387936 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5838d68c-4109-4d06-87dd-42ca2322bf0c admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:06.396383 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb70cd-d7bf-4133-9e66-cff8d584d363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:06.397700 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb70cd-d7bf-4133-9e66-cff8d584d363 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:06.398114 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb70cd-d7bf-4133-9e66-cff8d584d363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:06.399046 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb70cd-d7bf-4133-9e66-cff8d584d363 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:06.400154 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb70cd-d7bf-4133-9e66-cff8d584d363 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:06.403681 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb70cd-d7bf-4133-9e66-cff8d584d363 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:06.427827 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1130/2296] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:33:06 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:33:06.433815 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2578a5-1f3c-4bf4-854a-7117df64e74b admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:06.447146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2578a5-1f3c-4bf4-854a-7117df64e74b admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:06.454747 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bf8b51-a8bf-4067-b647-501089b748b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:06.456416 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf8b51-a8bf-4067-b647-501089b748b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:06.457022 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf8b51-a8bf-4067-b647-501089b748b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:06.457607 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf8b51-a8bf-4067-b647-501089b748b4 admin admin] PATH_INFO: `/v3/projects/9c3b0ccd53644a2994b88922b5d3ab70/users/7372f335b93a4a689f4d6f8350281840/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:06.489437 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bf8b51-a8bf-4067-b647-501089b748b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c3b0ccd53644a2994b88922b5d3ab70, user_id=7372f335b93a4a689f4d6f8350281840, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:06.492996 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bf8b51-a8bf-4067-b647-501089b748b4 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:06.515109 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1167/2297] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:33:06 2022] PUT /identity/v3/projects/9c3b0ccd53644a2994b88922b5d3ab70/users/7372f335b93a4a689f4d6f8350281840/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:06.527678 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd809a3a-2af9-41f7-b7f5-7bbae2c87adb None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:06.527678 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd809a3a-2af9-41f7-b7f5-7bbae2c87adb None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:06.527678 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd809a3a-2af9-41f7-b7f5-7bbae2c87adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:06.572254 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-cd809a3a-2af9-41f7-b7f5-7bbae2c87adb None None] MFA Rules not processed for user `7372f335b93a4a689f4d6f8350281840`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:06.574518 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-cd809a3a-2af9-41f7-b7f5-7bbae2c87adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:06.672160 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1131/2298] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:33:06 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:33:06.679137 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bf8b51-a8bf-4067-b647-501089b748b4 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:06.761471 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bf8b51-a8bf-4067-b647-501089b748b4 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:06.769263 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c4ff8-f225-438a-8293-9dc2a110e0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:06.770091 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c4ff8-f225-438a-8293-9dc2a110e0ef admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:06.770377 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c4ff8-f225-438a-8293-9dc2a110e0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:06.770655 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c4ff8-f225-438a-8293-9dc2a110e0ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:06.771372 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5c4ff8-f225-438a-8293-9dc2a110e0ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:06.773364 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5c4ff8-f225-438a-8293-9dc2a110e0ef admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:06.812216 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1168/2299] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:33:06 2022] POST /identity/v3/projects => generated 407 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:33:06.819067 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd809a3a-2af9-41f7-b7f5-7bbae2c87adb None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:06.835350 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd809a3a-2af9-41f7-b7f5-7bbae2c87adb None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:06.892910 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:06.956409 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719d78f0-0002-4197-bf06-2ec015915a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:06.958035 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d78f0-0002-4197-bf06-2ec015915a1e admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:06.958207 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d78f0-0002-4197-bf06-2ec015915a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:06.958665 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d78f0-0002-4197-bf06-2ec015915a1e admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:06.960093 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719d78f0-0002-4197-bf06-2ec015915a1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:06.964284 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719d78f0-0002-4197-bf06-2ec015915a1e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:06.998566 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1132/2300] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:33:06 2022] POST /identity/v3/users => generated 310 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:33:07.006164 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c4ff8-f225-438a-8293-9dc2a110e0ef admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:07.022203 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c4ff8-f225-438a-8293-9dc2a110e0ef admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:07.027830 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f625d8-8855-40a0-a977-e6c26bb84027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:07.028609 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f625d8-8855-40a0-a977-e6c26bb84027 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:07.029787 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f625d8-8855-40a0-a977-e6c26bb84027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:07.030133 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f625d8-8855-40a0-a977-e6c26bb84027 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:07.030997 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f625d8-8855-40a0-a977-e6c26bb84027 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:07.032853 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f625d8-8855-40a0-a977-e6c26bb84027 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:07.046058 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1169/2301] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:33:07.050834 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719d78f0-0002-4197-bf06-2ec015915a1e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:07.064271 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719d78f0-0002-4197-bf06-2ec015915a1e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:07.070087 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e8969-4454-49fe-b1ee-b1c38e752d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:07.072154 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e8969-4454-49fe-b1ee-b1c38e752d5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:07.072515 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e8969-4454-49fe-b1ee-b1c38e752d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:07.072859 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e8969-4454-49fe-b1ee-b1c38e752d5e admin admin] PATH_INFO: `/v3/projects/8a817a88b08c4f14b2dcbc602b0ababd/users/01a98ce2f0a84e399a401a4e16abaefb/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:07.095027 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435e8969-4454-49fe-b1ee-b1c38e752d5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a817a88b08c4f14b2dcbc602b0ababd, user_id=01a98ce2f0a84e399a401a4e16abaefb, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:07.099277 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435e8969-4454-49fe-b1ee-b1c38e752d5e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:07.127647 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1133/2302] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:33:07 2022] PUT /identity/v3/projects/8a817a88b08c4f14b2dcbc602b0ababd/users/01a98ce2f0a84e399a401a4e16abaefb/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:07.129704 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f625d8-8855-40a0-a977-e6c26bb84027 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:07.146371 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f625d8-8855-40a0-a977-e6c26bb84027 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:07.243019 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47233076-b34d-4f06-9b62-de0615b193dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:07.244140 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47233076-b34d-4f06-9b62-de0615b193dd admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:07.244569 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47233076-b34d-4f06-9b62-de0615b193dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:07.244916 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47233076-b34d-4f06-9b62-de0615b193dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:07.245787 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47233076-b34d-4f06-9b62-de0615b193dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:07.247645 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47233076-b34d-4f06-9b62-de0615b193dd admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:07.260176 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1170/2303] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:33:07.266675 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e8969-4454-49fe-b1ee-b1c38e752d5e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:07.279642 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e8969-4454-49fe-b1ee-b1c38e752d5e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:07.286175 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b052c-72c6-4944-94cf-fbf6c2a5666f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:07.287399 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b052c-72c6-4944-94cf-fbf6c2a5666f admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:07.287778 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b052c-72c6-4944-94cf-fbf6c2a5666f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:07.288155 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b052c-72c6-4944-94cf-fbf6c2a5666f admin admin] PATH_INFO: `/v3/projects/8a817a88b08c4f14b2dcbc602b0ababd/users/01a98ce2f0a84e399a401a4e16abaefb/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:07.292065 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5b052c-72c6-4944-94cf-fbf6c2a5666f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a817a88b08c4f14b2dcbc602b0ababd, user_id=01a98ce2f0a84e399a401a4e16abaefb, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:07.296039 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5b052c-72c6-4944-94cf-fbf6c2a5666f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:07.314728 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1134/2304] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:33:07 2022] PUT /identity/v3/projects/8a817a88b08c4f14b2dcbc602b0ababd/users/01a98ce2f0a84e399a401a4e16abaefb/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:07.328980 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07856842-6654-4c7e-9683-4d4112d1f07b None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:07.330106 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07856842-6654-4c7e-9683-4d4112d1f07b None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:07.330512 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07856842-6654-4c7e-9683-4d4112d1f07b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:07.340494 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:07.379816 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-07856842-6654-4c7e-9683-4d4112d1f07b None None] MFA Rules not processed for user `01a98ce2f0a84e399a401a4e16abaefb`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:07.381402 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-07856842-6654-4c7e-9683-4d4112d1f07b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:07.490225 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1171/2305] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:33:07 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:33:07.504333 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b052c-72c6-4944-94cf-fbf6c2a5666f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:07.506619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4d5b052c-72c6-4944-94cf-fbf6c2a5666f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:07.575498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b052c-72c6-4944-94cf-fbf6c2a5666f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:07.580710 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e724b307-ab74-4d94-9608-64e724399246 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:07.582172 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e724b307-ab74-4d94-9608-64e724399246 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:07.582680 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e724b307-ab74-4d94-9608-64e724399246 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:07.583112 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e724b307-ab74-4d94-9608-64e724399246 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:07.602958 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e724b307-ab74-4d94-9608-64e724399246 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:07.604927 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e724b307-ab74-4d94-9608-64e724399246 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:07.619111 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1135/2306] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:33:07 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:33:07.879529 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07856842-6654-4c7e-9683-4d4112d1f07b None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:07.891317 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07856842-6654-4c7e-9683-4d4112d1f07b None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:07.896771 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c94c30b-4992-44eb-8522-210451376575 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:07.899115 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94c30b-4992-44eb-8522-210451376575 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:07.899115 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94c30b-4992-44eb-8522-210451376575 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:07.899115 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94c30b-4992-44eb-8522-210451376575 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:07.985589 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c94c30b-4992-44eb-8522-210451376575 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:07.988310 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c94c30b-4992-44eb-8522-210451376575 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:08.007044 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1172/2307] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:33:07 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:33:16.795894 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8dafc5-274b-41bb-bc2c-05343fb5f954 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:16.796670 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8dafc5-274b-41bb-bc2c-05343fb5f954 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:16.796670 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8dafc5-274b-41bb-bc2c-05343fb5f954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:16.899780 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-ff8dafc5-274b-41bb-bc2c-05343fb5f954 None None] MFA Rules not processed for user `09c9fa7dccb8491c9ad3b43967345ea2`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:16.901116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ff8dafc5-274b-41bb-bc2c-05343fb5f954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:16.990375 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:17.008369 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1136/2308] 10.222.0.87 () {58 vars in 1009 bytes} [Tue Jun 21 18:33:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:33:17.107832 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c94c30b-4992-44eb-8522-210451376575 service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:17.112628 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8c94c30b-4992-44eb-8522-210451376575 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:17.325753 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c94c30b-4992-44eb-8522-210451376575 service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:17.334496 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d0a4ca-ae9b-4faa-927b-61154012f635 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '515b3571045e4fe29565307c8ecc10b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:17.335605 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d0a4ca-ae9b-4faa-927b-61154012f635 service swift] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:17.336265 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d0a4ca-ae9b-4faa-927b-61154012f635 service swift] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:17.336757 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d0a4ca-ae9b-4faa-927b-61154012f635 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:17.352319 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d0a4ca-ae9b-4faa-927b-61154012f635 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:17.355574 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d0a4ca-ae9b-4faa-927b-61154012f635 service swift] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:17.362479 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:17.387644 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1173/2309] 10.222.0.87 () {60 vars in 1384 bytes} [Tue Jun 21 18:33:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 21 18:33:27.010425 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:27.378346 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:35.069824 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8dafc5-274b-41bb-bc2c-05343fb5f954 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:35.072623 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ff8dafc5-274b-41bb-bc2c-05343fb5f954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:35.236261 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8dafc5-274b-41bb-bc2c-05343fb5f954 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:35.249266 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4736ce-64fd-4767-a347-06b5e32eeb7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:35.249266 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4736ce-64fd-4767-a347-06b5e32eeb7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:35.249266 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4736ce-64fd-4767-a347-06b5e32eeb7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:35.249266 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4736ce-64fd-4767-a347-06b5e32eeb7d admin admin] PATH_INFO: `/v3/users/8be1b8289d7548aca203e02ddc8808bb` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:35.251349 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4736ce-64fd-4767-a347-06b5e32eeb7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8be1b8289d7548aca203e02ddc8808bb)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:35.253027 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4736ce-64fd-4767-a347-06b5e32eeb7d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:35.347821 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-4b4736ce-64fd-4767-a347-06b5e32eeb7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8be1b8289d7548aca203e02ddc8808bb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:35.361491 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-4b4736ce-64fd-4767-a347-06b5e32eeb7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8be1b8289d7548aca203e02ddc8808bb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:35.362428 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-4b4736ce-64fd-4767-a347-06b5e32eeb7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8be1b8289d7548aca203e02ddc8808bb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:35.363875 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-4b4736ce-64fd-4767-a347-06b5e32eeb7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8be1b8289d7548aca203e02ddc8808bb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:35.393859 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-4b4736ce-64fd-4767-a347-06b5e32eeb7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8be1b8289d7548aca203e02ddc8808bb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:35.426666 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1137/2310] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:33:35 2022] DELETE /identity/v3/users/8be1b8289d7548aca203e02ddc8808bb => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:35.435083 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372e712-9f56-4aa4-9544-0a8432853780 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:35.437261 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372e712-9f56-4aa4-9544-0a8432853780 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:35.438458 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372e712-9f56-4aa4-9544-0a8432853780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:35.476360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-a372e712-9f56-4aa4-9544-0a8432853780 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:35.479949 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a372e712-9f56-4aa4-9544-0a8432853780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:35.618138 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1174/2311] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:33:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:33:35.630670 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4736ce-64fd-4767-a347-06b5e32eeb7d admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:35.633595 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4b4736ce-64fd-4767-a347-06b5e32eeb7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:35.737911 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4736ce-64fd-4767-a347-06b5e32eeb7d admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:35.750862 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5a7841-1130-4a13-838f-d95469533a3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:35.752327 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a7841-1130-4a13-838f-d95469533a3e service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:35.752842 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a7841-1130-4a13-838f-d95469533a3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:35.753589 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a7841-1130-4a13-838f-d95469533a3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:35.779021 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5a7841-1130-4a13-838f-d95469533a3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:35.781140 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5a7841-1130-4a13-838f-d95469533a3e service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:35.827510 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1138/2312] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:33:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:33:36.251926 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a372e712-9f56-4aa4-9544-0a8432853780 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:36.253995 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a372e712-9f56-4aa4-9544-0a8432853780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:36.273140 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a372e712-9f56-4aa4-9544-0a8432853780 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:36.290116 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ab28f5-4a26-4eab-b616-dabde75b74db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:36.291287 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab28f5-4a26-4eab-b616-dabde75b74db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:36.291856 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab28f5-4a26-4eab-b616-dabde75b74db admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:36.292230 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab28f5-4a26-4eab-b616-dabde75b74db admin admin] PATH_INFO: `/v3/users/7372f335b93a4a689f4d6f8350281840` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:36.293794 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ab28f5-4a26-4eab-b616-dabde75b74db admin admin] RBAC: Authorizing `identity:delete_user(user_id=7372f335b93a4a689f4d6f8350281840)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:36.295906 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ab28f5-4a26-4eab-b616-dabde75b74db admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:36.383885 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-55ab28f5-4a26-4eab-b616-dabde75b74db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7372f335b93a4a689f4d6f8350281840'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:36.405722 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-55ab28f5-4a26-4eab-b616-dabde75b74db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7372f335b93a4a689f4d6f8350281840'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:36.406532 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-55ab28f5-4a26-4eab-b616-dabde75b74db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7372f335b93a4a689f4d6f8350281840'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:36.407319 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-55ab28f5-4a26-4eab-b616-dabde75b74db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7372f335b93a4a689f4d6f8350281840'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:36.419321 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-55ab28f5-4a26-4eab-b616-dabde75b74db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7372f335b93a4a689f4d6f8350281840'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:36.444281 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1175/2313] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:33:36 2022] DELETE /identity/v3/users/7372f335b93a4a689f4d6f8350281840 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:36.454922 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0838fde-ff4c-4e12-968e-824db621ad88 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:36.455900 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0838fde-ff4c-4e12-968e-824db621ad88 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:36.456374 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0838fde-ff4c-4e12-968e-824db621ad88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:36.495288 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-e0838fde-ff4c-4e12-968e-824db621ad88 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:36.497875 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e0838fde-ff4c-4e12-968e-824db621ad88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:36.544905 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:36.656819 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ab28f5-4a26-4eab-b616-dabde75b74db admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:36.658416 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-55ab28f5-4a26-4eab-b616-dabde75b74db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:36.674298 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1139/2314] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:33:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:33:36.681439 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ab28f5-4a26-4eab-b616-dabde75b74db admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:36.699885 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0838fde-ff4c-4e12-968e-824db621ad88 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:36.702166 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e0838fde-ff4c-4e12-968e-824db621ad88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:36.703835 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9041ab9c-7146-4483-9284-107b730746fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:36.705088 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9041ab9c-7146-4483-9284-107b730746fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:36.706687 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9041ab9c-7146-4483-9284-107b730746fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:36.707238 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9041ab9c-7146-4483-9284-107b730746fc admin admin] PATH_INFO: `/v3/projects/1062e366f2224ea380ef02b5a61bfcd5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:36.710926 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9041ab9c-7146-4483-9284-107b730746fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=1062e366f2224ea380ef02b5a61bfcd5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:36.714540 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9041ab9c-7146-4483-9284-107b730746fc admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:36.837885 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9041ab9c-7146-4483-9284-107b730746fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1062e366f2224ea380ef02b5a61bfcd5', 'request_id': 'req-9041ab9c-7146-4483-9284-107b730746fc'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:36.864890 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0838fde-ff4c-4e12-968e-824db621ad88 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:36.874771 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9041ab9c-7146-4483-9284-107b730746fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1062e366f2224ea380ef02b5a61bfcd5', 'request_id': 'req-9041ab9c-7146-4483-9284-107b730746fc'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:36.891701 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9041ab9c-7146-4483-9284-107b730746fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1062e366f2224ea380ef02b5a61bfcd5', 'request_id': 'req-9041ab9c-7146-4483-9284-107b730746fc'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:36.907501 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:36.947685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9041ab9c-7146-4483-9284-107b730746fc admin admin] The token cache is being invalidate because project 1062e366f2224ea380ef02b5a61bfcd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:33:36.948370 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9041ab9c-7146-4483-9284-107b730746fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:36.950185 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9041ab9c-7146-4483-9284-107b730746fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:36.954964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1176/2315] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:33:36 2022] DELETE /identity/v3/projects/1062e366f2224ea380ef02b5a61bfcd5 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:33:36.961333 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a97987c-1615-475e-8a3b-23792ae4e4ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:36.962309 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a97987c-1615-475e-8a3b-23792ae4e4ad service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:36.962613 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a97987c-1615-475e-8a3b-23792ae4e4ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:36.962944 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a97987c-1615-475e-8a3b-23792ae4e4ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:36.965979 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3a97987c-1615-475e-8a3b-23792ae4e4ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:37.075595 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a97987c-1615-475e-8a3b-23792ae4e4ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:37.077471 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a97987c-1615-475e-8a3b-23792ae4e4ad service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:37.093310 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1140/2316] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:33:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:33:37.927400 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9041ab9c-7146-4483-9284-107b730746fc admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:37.931023 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-9041ab9c-7146-4483-9284-107b730746fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:37.959351 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9041ab9c-7146-4483-9284-107b730746fc admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:37.979346 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33684926-5d73-4b26-aba2-fb8f487f3888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:37.982474 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33684926-5d73-4b26-aba2-fb8f487f3888 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:37.983241 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33684926-5d73-4b26-aba2-fb8f487f3888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:37.983880 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33684926-5d73-4b26-aba2-fb8f487f3888 admin admin] PATH_INFO: `/v3/projects/9c3b0ccd53644a2994b88922b5d3ab70` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:37.986937 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33684926-5d73-4b26-aba2-fb8f487f3888 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c3b0ccd53644a2994b88922b5d3ab70)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:37.991449 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33684926-5d73-4b26-aba2-fb8f487f3888 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:38.110971 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-33684926-5d73-4b26-aba2-fb8f487f3888 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c3b0ccd53644a2994b88922b5d3ab70', 'request_id': 'req-33684926-5d73-4b26-aba2-fb8f487f3888'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:38.132753 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-33684926-5d73-4b26-aba2-fb8f487f3888 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c3b0ccd53644a2994b88922b5d3ab70', 'request_id': 'req-33684926-5d73-4b26-aba2-fb8f487f3888'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:38.156820 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-33684926-5d73-4b26-aba2-fb8f487f3888 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c3b0ccd53644a2994b88922b5d3ab70', 'request_id': 'req-33684926-5d73-4b26-aba2-fb8f487f3888'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:38.190630 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-33684926-5d73-4b26-aba2-fb8f487f3888 admin admin] The token cache is being invalidate because project 9c3b0ccd53644a2994b88922b5d3ab70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:33:38.191203 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-33684926-5d73-4b26-aba2-fb8f487f3888 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:38.192350 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-33684926-5d73-4b26-aba2-fb8f487f3888 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:38.196551 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1177/2317] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:33:37 2022] DELETE /identity/v3/projects/9c3b0ccd53644a2994b88922b5d3ab70 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:38.206035 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a97987c-1615-475e-8a3b-23792ae4e4ad service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:38.221277 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a97987c-1615-475e-8a3b-23792ae4e4ad service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:38.268839 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:38.341631 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12833035-03fe-4403-8f91-1f2e36b2b842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:38.343272 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12833035-03fe-4403-8f91-1f2e36b2b842 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:38.343784 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12833035-03fe-4403-8f91-1f2e36b2b842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:38.344271 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12833035-03fe-4403-8f91-1f2e36b2b842 admin admin] PATH_INFO: `/v3/users/01a98ce2f0a84e399a401a4e16abaefb` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:38.346642 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12833035-03fe-4403-8f91-1f2e36b2b842 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01a98ce2f0a84e399a401a4e16abaefb)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:38.351187 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12833035-03fe-4403-8f91-1f2e36b2b842 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:38.452241 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-12833035-03fe-4403-8f91-1f2e36b2b842 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01a98ce2f0a84e399a401a4e16abaefb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:38.476973 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-12833035-03fe-4403-8f91-1f2e36b2b842 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01a98ce2f0a84e399a401a4e16abaefb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:38.479233 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-12833035-03fe-4403-8f91-1f2e36b2b842 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01a98ce2f0a84e399a401a4e16abaefb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:38.481843 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-12833035-03fe-4403-8f91-1f2e36b2b842 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01a98ce2f0a84e399a401a4e16abaefb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:38.539683 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-12833035-03fe-4403-8f91-1f2e36b2b842 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01a98ce2f0a84e399a401a4e16abaefb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:38.575667 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1141/2318] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:33:38 2022] DELETE /identity/v3/users/01a98ce2f0a84e399a401a4e16abaefb => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:33:39.455567 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33684926-5d73-4b26-aba2-fb8f487f3888 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:39.459540 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-33684926-5d73-4b26-aba2-fb8f487f3888 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:39.500902 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33684926-5d73-4b26-aba2-fb8f487f3888 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:39.803826 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec541e4-e971-4d28-beb1-93ea02c329cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:39.806219 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec541e4-e971-4d28-beb1-93ea02c329cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:39.806960 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec541e4-e971-4d28-beb1-93ea02c329cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:39.807571 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec541e4-e971-4d28-beb1-93ea02c329cb admin admin] PATH_INFO: `/v3/projects/8a817a88b08c4f14b2dcbc602b0ababd` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:39.813158 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec541e4-e971-4d28-beb1-93ea02c329cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a817a88b08c4f14b2dcbc602b0ababd)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:39.823281 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec541e4-e971-4d28-beb1-93ea02c329cb admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:39.932486 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1ec541e4-e971-4d28-beb1-93ea02c329cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a817a88b08c4f14b2dcbc602b0ababd', 'request_id': 'req-1ec541e4-e971-4d28-beb1-93ea02c329cb'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:39.948946 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1ec541e4-e971-4d28-beb1-93ea02c329cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a817a88b08c4f14b2dcbc602b0ababd', 'request_id': 'req-1ec541e4-e971-4d28-beb1-93ea02c329cb'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:39.966185 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1ec541e4-e971-4d28-beb1-93ea02c329cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a817a88b08c4f14b2dcbc602b0ababd', 'request_id': 'req-1ec541e4-e971-4d28-beb1-93ea02c329cb'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:39.996079 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1ec541e4-e971-4d28-beb1-93ea02c329cb admin admin] The token cache is being invalidate because project 8a817a88b08c4f14b2dcbc602b0ababd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:33:39.996563 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1ec541e4-e971-4d28-beb1-93ea02c329cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:39.997904 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1ec541e4-e971-4d28-beb1-93ea02c329cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:40.007558 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1178/2319] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:33:39 2022] DELETE /identity/v3/projects/8a817a88b08c4f14b2dcbc602b0ababd => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:40.010378 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f64a6-e1ff-46fa-9b23-9dcc3f4bbf82 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:40.011307 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f64a6-e1ff-46fa-9b23-9dcc3f4bbf82 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:40.011448 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f64a6-e1ff-46fa-9b23-9dcc3f4bbf82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:40.058880 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f30f64a6-e1ff-46fa-9b23-9dcc3f4bbf82 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:40.064302 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f30f64a6-e1ff-46fa-9b23-9dcc3f4bbf82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:40.251572 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1142/2320] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:33:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:33:40.256231 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6af99c-bdea-4c1f-b641-9a675673094c None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:40.257262 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6af99c-bdea-4c1f-b641-9a675673094c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:40.258149 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6af99c-bdea-4c1f-b641-9a675673094c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:40.311404 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-5b6af99c-bdea-4c1f-b641-9a675673094c None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:40.314315 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5b6af99c-bdea-4c1f-b641-9a675673094c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:40.342563 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1179/2321] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:33:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:33:40.346971 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f64a6-e1ff-46fa-9b23-9dcc3f4bbf82 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:40.368084 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f64a6-e1ff-46fa-9b23-9dcc3f4bbf82 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:40.383024 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e810ac6-0f17-4843-9188-fb28c8900e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:40.384412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e810ac6-0f17-4843-9188-fb28c8900e48 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:40.384726 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e810ac6-0f17-4843-9188-fb28c8900e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:40.385274 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e810ac6-0f17-4843-9188-fb28c8900e48 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:40.386885 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e810ac6-0f17-4843-9188-fb28c8900e48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:33:40.387652 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e810ac6-0f17-4843-9188-fb28c8900e48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:40.391444 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e810ac6-0f17-4843-9188-fb28c8900e48 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:40.424918 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1143/2322] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:33:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:33:40.427384 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6af99c-bdea-4c1f-b641-9a675673094c None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:40.431928 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:40.440601 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6af99c-bdea-4c1f-b641-9a675673094c None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:40.447515 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c18a0-50b3-49a2-b42e-c8c5997c9917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:40.448369 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c18a0-50b3-49a2-b42e-c8c5997c9917 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:40.449394 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c18a0-50b3-49a2-b42e-c8c5997c9917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:40.449785 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c18a0-50b3-49a2-b42e-c8c5997c9917 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:40.451287 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524c18a0-50b3-49a2-b42e-c8c5997c9917 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:40.454081 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524c18a0-50b3-49a2-b42e-c8c5997c9917 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:40.507249 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1180/2323] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:33:40 2022] POST /identity/v3/projects => generated 379 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:33:40.510467 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e810ac6-0f17-4843-9188-fb28c8900e48 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:40.542188 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e810ac6-0f17-4843-9188-fb28c8900e48 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:40.605101 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:40.651636 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2d538e-3c3c-48b3-a8af-854614040519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:40.652553 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d538e-3c3c-48b3-a8af-854614040519 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:40.652939 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d538e-3c3c-48b3-a8af-854614040519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:40.653314 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d538e-3c3c-48b3-a8af-854614040519 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:40.654226 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2d538e-3c3c-48b3-a8af-854614040519 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:40.656444 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2d538e-3c3c-48b3-a8af-854614040519 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:40.695576 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1144/2324] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:33:40 2022] POST /identity/v3/users => generated 290 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:33:40.701005 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c18a0-50b3-49a2-b42e-c8c5997c9917 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:40.720487 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c18a0-50b3-49a2-b42e-c8c5997c9917 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:40.729504 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8b885f-80fb-410a-bbfe-34bac4cdb535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:40.730687 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b885f-80fb-410a-bbfe-34bac4cdb535 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:40.731187 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b885f-80fb-410a-bbfe-34bac4cdb535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:40.731730 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b885f-80fb-410a-bbfe-34bac4cdb535 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:40.733226 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8b885f-80fb-410a-bbfe-34bac4cdb535 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:40.736468 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8b885f-80fb-410a-bbfe-34bac4cdb535 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:40.753513 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1181/2325] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:33:40.759899 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2d538e-3c3c-48b3-a8af-854614040519 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:40.778585 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2d538e-3c3c-48b3-a8af-854614040519 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:40.787351 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3d8d9-a5de-4298-a178-7b3d77c74e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:40.788653 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3d8d9-a5de-4298-a178-7b3d77c74e0d admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:40.789332 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3d8d9-a5de-4298-a178-7b3d77c74e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:40.789693 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3d8d9-a5de-4298-a178-7b3d77c74e0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:40.790697 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d3d8d9-a5de-4298-a178-7b3d77c74e0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:40.792923 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d3d8d9-a5de-4298-a178-7b3d77c74e0d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:40.809725 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1145/2326] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:33:40.814618 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8b885f-80fb-410a-bbfe-34bac4cdb535 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:40.851561 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8b885f-80fb-410a-bbfe-34bac4cdb535 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:40.861144 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98684ea9-38b1-4622-a438-6e7cc366bb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:40.862712 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98684ea9-38b1-4622-a438-6e7cc366bb27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:40.863191 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98684ea9-38b1-4622-a438-6e7cc366bb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:40.863666 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98684ea9-38b1-4622-a438-6e7cc366bb27 admin admin] PATH_INFO: `/v3/projects/b49d6e0d43eb4c1bb85fb49bc7a0d77b/users/32a9b26ce06f497f987f58bd430cd6b7/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:40.894689 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98684ea9-38b1-4622-a438-6e7cc366bb27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b49d6e0d43eb4c1bb85fb49bc7a0d77b, user_id=32a9b26ce06f497f987f58bd430cd6b7, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:40.900121 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98684ea9-38b1-4622-a438-6e7cc366bb27 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:40.921533 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1182/2327] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:33:40 2022] PUT /identity/v3/projects/b49d6e0d43eb4c1bb85fb49bc7a0d77b/users/32a9b26ce06f497f987f58bd430cd6b7/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:40.942292 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cda01e-9950-46f9-bfe2-c409195e44cc None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:40.942816 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cda01e-9950-46f9-bfe2-c409195e44cc None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:40.943263 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cda01e-9950-46f9-bfe2-c409195e44cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:40.997291 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b0cda01e-9950-46f9-bfe2-c409195e44cc None None] MFA Rules not processed for user `32a9b26ce06f497f987f58bd430cd6b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:40.999547 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b0cda01e-9950-46f9-bfe2-c409195e44cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:41.125725 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1146/2328] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:33:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:33:41.131340 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98684ea9-38b1-4622-a438-6e7cc366bb27 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:41.229805 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98684ea9-38b1-4622-a438-6e7cc366bb27 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:41.237320 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a8bf60-afcf-4551-879c-0df63fa13ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:41.238430 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a8bf60-afcf-4551-879c-0df63fa13ec5 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:41.238906 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a8bf60-afcf-4551-879c-0df63fa13ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:41.239382 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a8bf60-afcf-4551-879c-0df63fa13ec5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:41.240705 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a8bf60-afcf-4551-879c-0df63fa13ec5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:41.244890 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a8bf60-afcf-4551-879c-0df63fa13ec5 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:41.289855 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1183/2329] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:33:41 2022] POST /identity/v3/projects => generated 377 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:33:41.293477 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cda01e-9950-46f9-bfe2-c409195e44cc None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:41.317403 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cda01e-9950-46f9-bfe2-c409195e44cc None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:41.423133 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a195f-3e97-4de2-bdc8-848c22309d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:41.423862 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a195f-3e97-4de2-bdc8-848c22309d2d admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:41.424339 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a195f-3e97-4de2-bdc8-848c22309d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:41.424789 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a195f-3e97-4de2-bdc8-848c22309d2d admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:41.425784 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4a195f-3e97-4de2-bdc8-848c22309d2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:41.428567 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4a195f-3e97-4de2-bdc8-848c22309d2d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:41.466547 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1147/2330] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:33:41 2022] POST /identity/v3/users => generated 289 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:33:41.467908 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a8bf60-afcf-4551-879c-0df63fa13ec5 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:41.484896 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a8bf60-afcf-4551-879c-0df63fa13ec5 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:41.494481 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3df788d-c0e1-46f3-b098-4cb5b7ba3975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:41.495436 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df788d-c0e1-46f3-b098-4cb5b7ba3975 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:41.495853 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df788d-c0e1-46f3-b098-4cb5b7ba3975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:41.496232 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df788d-c0e1-46f3-b098-4cb5b7ba3975 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:41.497169 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3df788d-c0e1-46f3-b098-4cb5b7ba3975 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:41.499005 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3df788d-c0e1-46f3-b098-4cb5b7ba3975 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:41.513773 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1184/2331] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:33:41.520289 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a195f-3e97-4de2-bdc8-848c22309d2d admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:41.539786 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a195f-3e97-4de2-bdc8-848c22309d2d admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:41.549074 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364ce82-e77c-4990-9caf-1cd976126d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:41.550304 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364ce82-e77c-4990-9caf-1cd976126d2d admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:41.550796 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364ce82-e77c-4990-9caf-1cd976126d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:41.551289 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364ce82-e77c-4990-9caf-1cd976126d2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:41.553091 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2364ce82-e77c-4990-9caf-1cd976126d2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:41.556430 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2364ce82-e77c-4990-9caf-1cd976126d2d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:41.582587 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1148/2332] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:33:41.593493 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3df788d-c0e1-46f3-b098-4cb5b7ba3975 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:41.612305 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3df788d-c0e1-46f3-b098-4cb5b7ba3975 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:41.620748 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af131132-0615-4b80-89c6-06333715faec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:41.622433 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af131132-0615-4b80-89c6-06333715faec admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:41.622801 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af131132-0615-4b80-89c6-06333715faec admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:41.623289 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af131132-0615-4b80-89c6-06333715faec admin admin] PATH_INFO: `/v3/projects/b62ca666f8794ae6bc4b5b7601cbde4b/users/233b67c192d04043b2fc4c0ceeb07776/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:41.647584 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af131132-0615-4b80-89c6-06333715faec admin admin] RBAC: Authorizing `identity:create_grant(project_id=b62ca666f8794ae6bc4b5b7601cbde4b, user_id=233b67c192d04043b2fc4c0ceeb07776, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:41.650727 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af131132-0615-4b80-89c6-06333715faec admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:41.669520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1185/2333] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:33:41 2022] PUT /identity/v3/projects/b62ca666f8794ae6bc4b5b7601cbde4b/users/233b67c192d04043b2fc4c0ceeb07776/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:33:41.689420 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0155e3a-f6d6-47bb-bd6a-22cf46259086 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:41.689977 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0155e3a-f6d6-47bb-bd6a-22cf46259086 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:41.690468 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0155e3a-f6d6-47bb-bd6a-22cf46259086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:41.738012 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f0155e3a-f6d6-47bb-bd6a-22cf46259086 None None] MFA Rules not processed for user `233b67c192d04043b2fc4c0ceeb07776`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:41.739469 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f0155e3a-f6d6-47bb-bd6a-22cf46259086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:41.755525 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:41.851859 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1149/2334] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:33:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:33:41.857255 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af131132-0615-4b80-89c6-06333715faec admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:41.965782 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:41.972473 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af131132-0615-4b80-89c6-06333715faec admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:41.981748 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d45d20-6a4e-4101-9be5-91181b68af8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:41.982874 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d45d20-6a4e-4101-9be5-91181b68af8f admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:41.983241 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d45d20-6a4e-4101-9be5-91181b68af8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:41.983581 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d45d20-6a4e-4101-9be5-91181b68af8f admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:41.984414 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d45d20-6a4e-4101-9be5-91181b68af8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:41.987877 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d45d20-6a4e-4101-9be5-91181b68af8f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:42.033357 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1186/2335] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:33:41 2022] POST /identity/v3/projects => generated 377 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:33:42.040032 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0155e3a-f6d6-47bb-bd6a-22cf46259086 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:42.053050 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0155e3a-f6d6-47bb-bd6a-22cf46259086 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:42.144614 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e75496-420f-4f41-afe2-7560949199d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:42.145763 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e75496-420f-4f41-afe2-7560949199d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:42.146151 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e75496-420f-4f41-afe2-7560949199d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:42.146532 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e75496-420f-4f41-afe2-7560949199d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:42.147464 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e75496-420f-4f41-afe2-7560949199d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:42.149605 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e75496-420f-4f41-afe2-7560949199d9 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:42.182924 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1150/2336] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:33:42 2022] POST /identity/v3/users => generated 295 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:33:42.187995 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d45d20-6a4e-4101-9be5-91181b68af8f admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:42.200273 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d45d20-6a4e-4101-9be5-91181b68af8f admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:42.207366 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c9e98f-e4df-447e-9c18-b51a1444abe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:42.208078 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c9e98f-e4df-447e-9c18-b51a1444abe4 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:42.208361 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c9e98f-e4df-447e-9c18-b51a1444abe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:42.208635 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c9e98f-e4df-447e-9c18-b51a1444abe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:42.209381 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c9e98f-e4df-447e-9c18-b51a1444abe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:42.211184 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c9e98f-e4df-447e-9c18-b51a1444abe4 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:42.224865 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1187/2337] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:33:42.231235 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e75496-420f-4f41-afe2-7560949199d9 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:42.249292 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e75496-420f-4f41-afe2-7560949199d9 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:42.257727 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ff72f-e00d-4e0e-967d-9d53ee6fcb02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:42.259188 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ff72f-e00d-4e0e-967d-9d53ee6fcb02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:42.259678 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ff72f-e00d-4e0e-967d-9d53ee6fcb02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:42.260105 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ff72f-e00d-4e0e-967d-9d53ee6fcb02 admin admin] PATH_INFO: `/v3/projects/fa935717fb134af4b7a4d0fe71940d20/users/451b1b59d40841f9b348db16d13c407d/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:42.289080 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3ff72f-e00d-4e0e-967d-9d53ee6fcb02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa935717fb134af4b7a4d0fe71940d20, user_id=451b1b59d40841f9b348db16d13c407d, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:42.294532 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3ff72f-e00d-4e0e-967d-9d53ee6fcb02 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:42.318053 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1151/2338] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:33:42 2022] PUT /identity/v3/projects/fa935717fb134af4b7a4d0fe71940d20/users/451b1b59d40841f9b348db16d13c407d/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:42.324685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c9e98f-e4df-447e-9c18-b51a1444abe4 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:42.343469 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c9e98f-e4df-447e-9c18-b51a1444abe4 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:42.438542 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a642703-8543-4f83-a432-d992246a727c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:42.439254 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a642703-8543-4f83-a432-d992246a727c admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:42.439593 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a642703-8543-4f83-a432-d992246a727c admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:42.439913 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a642703-8543-4f83-a432-d992246a727c admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:42.440798 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a642703-8543-4f83-a432-d992246a727c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:42.443078 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a642703-8543-4f83-a432-d992246a727c admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:42.457175 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1188/2339] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:33:42.463503 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ff72f-e00d-4e0e-967d-9d53ee6fcb02 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:42.476065 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ff72f-e00d-4e0e-967d-9d53ee6fcb02 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:42.482431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57de20fc-dbf7-4b53-9558-2a0c60afa773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:42.483488 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de20fc-dbf7-4b53-9558-2a0c60afa773 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:42.483877 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de20fc-dbf7-4b53-9558-2a0c60afa773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:42.484250 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de20fc-dbf7-4b53-9558-2a0c60afa773 admin admin] PATH_INFO: `/v3/projects/fa935717fb134af4b7a4d0fe71940d20/users/451b1b59d40841f9b348db16d13c407d/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:42.486736 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57de20fc-dbf7-4b53-9558-2a0c60afa773 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa935717fb134af4b7a4d0fe71940d20, user_id=451b1b59d40841f9b348db16d13c407d, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:42.489884 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57de20fc-dbf7-4b53-9558-2a0c60afa773 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:42.504175 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1152/2340] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:33:42 2022] PUT /identity/v3/projects/fa935717fb134af4b7a4d0fe71940d20/users/451b1b59d40841f9b348db16d13c407d/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:42.516175 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29344817-065e-4871-a997-9bb70c115bd2 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:42.517041 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29344817-065e-4871-a997-9bb70c115bd2 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:42.517396 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29344817-065e-4871-a997-9bb70c115bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:42.551720 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-29344817-065e-4871-a997-9bb70c115bd2 None None] MFA Rules not processed for user `451b1b59d40841f9b348db16d13c407d`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:42.552963 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-29344817-065e-4871-a997-9bb70c115bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:42.668927 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1189/2341] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:33:42 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:33:42.682858 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57de20fc-dbf7-4b53-9558-2a0c60afa773 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:42.685908 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-57de20fc-dbf7-4b53-9558-2a0c60afa773 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:42.807636 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f312906-6260-4ef0-8377-a175335c5814 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:42.808221 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f312906-6260-4ef0-8377-a175335c5814 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:42.808842 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f312906-6260-4ef0-8377-a175335c5814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:42.811387 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57de20fc-dbf7-4b53-9558-2a0c60afa773 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:42.815404 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e557430e-22e4-4e54-9788-171da21d150d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:42.816243 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e557430e-22e4-4e54-9788-171da21d150d service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:42.816520 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e557430e-22e4-4e54-9788-171da21d150d service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:42.816795 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e557430e-22e4-4e54-9788-171da21d150d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:42.833278 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e557430e-22e4-4e54-9788-171da21d150d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:42.837284 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e557430e-22e4-4e54-9788-171da21d150d service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:42.838174 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-4f312906-6260-4ef0-8377-a175335c5814 None None] MFA Rules not processed for user `09c9fa7dccb8491c9ad3b43967345ea2`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:42.840595 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4f312906-6260-4ef0-8377-a175335c5814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:42.857178 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1153/2342] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:33:42 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:33:42.918397 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e557430e-22e4-4e54-9788-171da21d150d service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:42.920666 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e557430e-22e4-4e54-9788-171da21d150d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:42.922865 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:42.935355 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e557430e-22e4-4e54-9788-171da21d150d service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:42.939717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec5ff8-2c23-4958-aa49-6cf0d1235759 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:42.940395 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec5ff8-2c23-4958-aa49-6cf0d1235759 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:42.940685 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec5ff8-2c23-4958-aa49-6cf0d1235759 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:42.940964 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec5ff8-2c23-4958-aa49-6cf0d1235759 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:42.942750 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1190/2343] 10.222.0.87 () {58 vars in 967 bytes} [Tue Jun 21 18:33:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:33:42.947339 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:42.957467 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f312906-6260-4ef0-8377-a175335c5814 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:42.959199 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4f312906-6260-4ef0-8377-a175335c5814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:43.057219 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec5ff8-2c23-4958-aa49-6cf0d1235759 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:43.059976 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec5ff8-2c23-4958-aa49-6cf0d1235759 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:43.078072 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f312906-6260-4ef0-8377-a175335c5814 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:43.080324 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1154/2344] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:33:42 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:33:43.102472 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dda622f-fba1-4fe5-98a2-31b26e449e1d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '515b3571045e4fe29565307c8ecc10b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:43.103681 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dda622f-fba1-4fe5-98a2-31b26e449e1d service swift] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:43.104176 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dda622f-fba1-4fe5-98a2-31b26e449e1d service swift] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:43.104668 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dda622f-fba1-4fe5-98a2-31b26e449e1d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:43.122422 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dda622f-fba1-4fe5-98a2-31b26e449e1d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:43.122422 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dda622f-fba1-4fe5-98a2-31b26e449e1d service swift] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:43.136071 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1191/2345] 10.222.0.87 () {60 vars in 1384 bytes} [Tue Jun 21 18:33:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 21 18:33:45.711576 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec5ff8-2c23-4958-aa49-6cf0d1235759 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:45.714456 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-02ec5ff8-2c23-4958-aa49-6cf0d1235759 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:45.827689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec5ff8-2c23-4958-aa49-6cf0d1235759 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:45.833064 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd8e583-c180-4687-8061-24d22dea314d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:45.834021 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8e583-c180-4687-8061-24d22dea314d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:45.835748 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8e583-c180-4687-8061-24d22dea314d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:45.835748 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8e583-c180-4687-8061-24d22dea314d admin admin] PATH_INFO: `/v3/users/82c323bc7a8c45e39714bb3e7f401c3a` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:45.838448 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd8e583-c180-4687-8061-24d22dea314d admin admin] RBAC: Authorizing `identity:delete_user(user_id=82c323bc7a8c45e39714bb3e7f401c3a)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:45.842539 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd8e583-c180-4687-8061-24d22dea314d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:45.906252 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3fd8e583-c180-4687-8061-24d22dea314d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82c323bc7a8c45e39714bb3e7f401c3a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:45.915934 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3fd8e583-c180-4687-8061-24d22dea314d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82c323bc7a8c45e39714bb3e7f401c3a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:45.916713 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3fd8e583-c180-4687-8061-24d22dea314d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82c323bc7a8c45e39714bb3e7f401c3a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:45.917469 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3fd8e583-c180-4687-8061-24d22dea314d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82c323bc7a8c45e39714bb3e7f401c3a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:45.933448 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3fd8e583-c180-4687-8061-24d22dea314d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82c323bc7a8c45e39714bb3e7f401c3a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:45.949747 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1155/2346] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:33:45 2022] DELETE /identity/v3/users/82c323bc7a8c45e39714bb3e7f401c3a => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:46.181547 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dda622f-fba1-4fe5-98a2-31b26e449e1d service swift] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:46.195966 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dda622f-fba1-4fe5-98a2-31b26e449e1d service swift] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:46.291455 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7f7e7d-291e-4727-8cab-8e508b081700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:46.292479 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f7e7d-291e-4727-8cab-8e508b081700 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:46.292799 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f7e7d-291e-4727-8cab-8e508b081700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:46.293157 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f7e7d-291e-4727-8cab-8e508b081700 admin admin] PATH_INFO: `/v3/projects/0307481a33be4b268d2793bb65f9d9ef` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:46.295751 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7f7e7d-291e-4727-8cab-8e508b081700 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0307481a33be4b268d2793bb65f9d9ef)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:46.297826 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7f7e7d-291e-4727-8cab-8e508b081700 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:46.359109 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-fd7f7e7d-291e-4727-8cab-8e508b081700 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0307481a33be4b268d2793bb65f9d9ef', 'request_id': 'req-fd7f7e7d-291e-4727-8cab-8e508b081700'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:46.366980 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-fd7f7e7d-291e-4727-8cab-8e508b081700 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0307481a33be4b268d2793bb65f9d9ef', 'request_id': 'req-fd7f7e7d-291e-4727-8cab-8e508b081700'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:46.378500 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-fd7f7e7d-291e-4727-8cab-8e508b081700 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0307481a33be4b268d2793bb65f9d9ef', 'request_id': 'req-fd7f7e7d-291e-4727-8cab-8e508b081700'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:46.402188 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-fd7f7e7d-291e-4727-8cab-8e508b081700 admin admin] The token cache is being invalidate because project 0307481a33be4b268d2793bb65f9d9ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:33:46.402484 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-fd7f7e7d-291e-4727-8cab-8e508b081700 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:46.403291 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-fd7f7e7d-291e-4727-8cab-8e508b081700 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:46.406023 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1192/2347] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:33:46 2022] DELETE /identity/v3/projects/0307481a33be4b268d2793bb65f9d9ef => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:33:46.414193 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b70dd2-4522-4017-9e54-6fb93954fb64 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:46.414540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b70dd2-4522-4017-9e54-6fb93954fb64 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:46.415047 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b70dd2-4522-4017-9e54-6fb93954fb64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:46.444558 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-66b70dd2-4522-4017-9e54-6fb93954fb64 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:46.446134 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-66b70dd2-4522-4017-9e54-6fb93954fb64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:46.492481 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:46.589488 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1156/2348] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:33:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:33:46.602206 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65f5fa-8c8d-4e68-ba1e-807d34e57ec7 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:46.602744 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65f5fa-8c8d-4e68-ba1e-807d34e57ec7 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:46.603047 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65f5fa-8c8d-4e68-ba1e-807d34e57ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:46.638739 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-bc65f5fa-8c8d-4e68-ba1e-807d34e57ec7 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:46.641564 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-bc65f5fa-8c8d-4e68-ba1e-807d34e57ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:46.657575 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1193/2349] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:33:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:33:46.659498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b70dd2-4522-4017-9e54-6fb93954fb64 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:46.675335 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b70dd2-4522-4017-9e54-6fb93954fb64 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:46.684226 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fdc89d-ed5d-467d-acbb-832045d39d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:46.685047 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fdc89d-ed5d-467d-acbb-832045d39d30 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:46.685192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fdc89d-ed5d-467d-acbb-832045d39d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:46.685470 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fdc89d-ed5d-467d-acbb-832045d39d30 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:46.686310 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fdc89d-ed5d-467d-acbb-832045d39d30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:33:46.686665 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fdc89d-ed5d-467d-acbb-832045d39d30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:46.688326 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fdc89d-ed5d-467d-acbb-832045d39d30 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:46.709686 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1157/2350] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:33:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:33:46.710283 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65f5fa-8c8d-4e68-ba1e-807d34e57ec7 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:46.715945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fdc89d-ed5d-467d-acbb-832045d39d30 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:46.718361 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e2fdc89d-ed5d-467d-acbb-832045d39d30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:46.718583 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65f5fa-8c8d-4e68-ba1e-807d34e57ec7 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:46.724147 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce08c665-24df-450e-9d05-a87f23757bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:46.724830 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce08c665-24df-450e-9d05-a87f23757bcc admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:46.725124 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce08c665-24df-450e-9d05-a87f23757bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:46.725401 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce08c665-24df-450e-9d05-a87f23757bcc admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:46.726193 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce08c665-24df-450e-9d05-a87f23757bcc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:46.728273 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce08c665-24df-450e-9d05-a87f23757bcc admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:46.734286 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fdc89d-ed5d-467d-acbb-832045d39d30 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:46.734590 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:46.740116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894edc09-1ad8-48c8-93f7-e28f379fb27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:46.741156 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894edc09-1ad8-48c8-93f7-e28f379fb27b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:46.741538 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894edc09-1ad8-48c8-93f7-e28f379fb27b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:46.741860 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894edc09-1ad8-48c8-93f7-e28f379fb27b admin admin] PATH_INFO: `/v3/users/32a9b26ce06f497f987f58bd430cd6b7` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:46.743448 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894edc09-1ad8-48c8-93f7-e28f379fb27b admin admin] RBAC: Authorizing `identity:delete_user(user_id=32a9b26ce06f497f987f58bd430cd6b7)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:46.745949 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894edc09-1ad8-48c8-93f7-e28f379fb27b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:46.770839 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1194/2351] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:33:46 2022] POST /identity/v3/projects => generated 369 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:33:46.772883 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce08c665-24df-450e-9d05-a87f23757bcc admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:46.807348 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-894edc09-1ad8-48c8-93f7-e28f379fb27b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32a9b26ce06f497f987f58bd430cd6b7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:46.827716 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-894edc09-1ad8-48c8-93f7-e28f379fb27b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32a9b26ce06f497f987f58bd430cd6b7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:46.828539 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-894edc09-1ad8-48c8-93f7-e28f379fb27b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32a9b26ce06f497f987f58bd430cd6b7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:46.829555 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-894edc09-1ad8-48c8-93f7-e28f379fb27b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32a9b26ce06f497f987f58bd430cd6b7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:46.854298 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-894edc09-1ad8-48c8-93f7-e28f379fb27b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32a9b26ce06f497f987f58bd430cd6b7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:46.868362 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce08c665-24df-450e-9d05-a87f23757bcc admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:46.876524 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56910f3a-1a14-4e8f-b8fa-4a38d0f12e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:46.877549 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56910f3a-1a14-4e8f-b8fa-4a38d0f12e71 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:46.878049 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56910f3a-1a14-4e8f-b8fa-4a38d0f12e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:46.878485 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56910f3a-1a14-4e8f-b8fa-4a38d0f12e71 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:46.879651 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56910f3a-1a14-4e8f-b8fa-4a38d0f12e71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:46.882019 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1158/2352] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:33:46 2022] DELETE /identity/v3/users/32a9b26ce06f497f987f58bd430cd6b7 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:46.882934 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56910f3a-1a14-4e8f-b8fa-4a38d0f12e71 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:46.887853 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feab7ab-a1f4-478d-b454-1c0696a957c3 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:46.888363 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feab7ab-a1f4-478d-b454-1c0696a957c3 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:46.889088 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feab7ab-a1f4-478d-b454-1c0696a957c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:46.919940 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-1feab7ab-a1f4-478d-b454-1c0696a957c3 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:46.921846 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1feab7ab-a1f4-478d-b454-1c0696a957c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:46.927106 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1195/2353] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:33:46 2022] POST /identity/v3/users => generated 285 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:33:46.932869 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56910f3a-1a14-4e8f-b8fa-4a38d0f12e71 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:46.935075 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-56910f3a-1a14-4e8f-b8fa-4a38d0f12e71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:46.953164 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56910f3a-1a14-4e8f-b8fa-4a38d0f12e71 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:47.021005 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1159/2354] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:33:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:33:47.036411 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feab7ab-a1f4-478d-b454-1c0696a957c3 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:47.038708 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1feab7ab-a1f4-478d-b454-1c0696a957c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:47.048354 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d542478e-0663-43c7-859f-23a31d4c5f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:47.049193 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d542478e-0663-43c7-859f-23a31d4c5f26 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:47.049765 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d542478e-0663-43c7-859f-23a31d4c5f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:47.050214 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d542478e-0663-43c7-859f-23a31d4c5f26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:47.051459 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d542478e-0663-43c7-859f-23a31d4c5f26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:47.054434 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d542478e-0663-43c7-859f-23a31d4c5f26 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:47.069881 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1196/2355] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:33:46 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:33:47.074135 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d542478e-0663-43c7-859f-23a31d4c5f26 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:47.082099 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:47.087007 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d542478e-0663-43c7-859f-23a31d4c5f26 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:47.093918 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8438743-da3c-4ebd-bc5e-1c0620f444cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:47.096183 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8438743-da3c-4ebd-bc5e-1c0620f444cd admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:47.096603 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8438743-da3c-4ebd-bc5e-1c0620f444cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:47.097102 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8438743-da3c-4ebd-bc5e-1c0620f444cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:47.098090 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8438743-da3c-4ebd-bc5e-1c0620f444cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:47.100409 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8438743-da3c-4ebd-bc5e-1c0620f444cd admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:47.116382 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1197/2356] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:33:47 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:33:47.122617 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8438743-da3c-4ebd-bc5e-1c0620f444cd admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:47.128826 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feab7ab-a1f4-478d-b454-1c0696a957c3 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:47.135319 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81362dec-5ec6-4412-8c5c-570e60c72b9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:47.136282 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81362dec-5ec6-4412-8c5c-570e60c72b9a service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:47.136743 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81362dec-5ec6-4412-8c5c-570e60c72b9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:47.137205 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81362dec-5ec6-4412-8c5c-570e60c72b9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:47.139214 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8438743-da3c-4ebd-bc5e-1c0620f444cd admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:47.146689 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89211d20-2751-49aa-b7a3-728dda41c2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:47.148099 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89211d20-2751-49aa-b7a3-728dda41c2c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:47.148593 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89211d20-2751-49aa-b7a3-728dda41c2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:47.149036 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89211d20-2751-49aa-b7a3-728dda41c2c4 admin admin] PATH_INFO: `/v3/projects/34c100a7ff2744348040ec3b057f3022/users/fa5c0c04ba20442caebee478f2b52c0d/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:47.154426 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81362dec-5ec6-4412-8c5c-570e60c72b9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:47.156601 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81362dec-5ec6-4412-8c5c-570e60c72b9a service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:47.172452 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89211d20-2751-49aa-b7a3-728dda41c2c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34c100a7ff2744348040ec3b057f3022, user_id=fa5c0c04ba20442caebee478f2b52c0d, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:47.173012 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1160/2357] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:33:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:33:47.177717 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89211d20-2751-49aa-b7a3-728dda41c2c4 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:47.201680 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1198/2358] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:33:47 2022] PUT /identity/v3/projects/34c100a7ff2744348040ec3b057f3022/users/fa5c0c04ba20442caebee478f2b52c0d/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:33:47.218281 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600af6c4-23ef-45f7-bd25-07692e378f06 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:47.218691 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600af6c4-23ef-45f7-bd25-07692e378f06 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:47.219071 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600af6c4-23ef-45f7-bd25-07692e378f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:47.265718 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-600af6c4-23ef-45f7-bd25-07692e378f06 None None] MFA Rules not processed for user `fa5c0c04ba20442caebee478f2b52c0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:47.267144 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-600af6c4-23ef-45f7-bd25-07692e378f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:47.366996 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1161/2359] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:33:47 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:33:47.382293 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89211d20-2751-49aa-b7a3-728dda41c2c4 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:47.385238 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-89211d20-2751-49aa-b7a3-728dda41c2c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:47.477368 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:47.516615 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89211d20-2751-49aa-b7a3-728dda41c2c4 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:47.523119 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc831d8-05ce-426f-b660-0f799d901608 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7da4bf6f3be47b4aa25c6240cb4be03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:47.524244 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc831d8-05ce-426f-b660-0f799d901608 service glance] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:47.524812 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc831d8-05ce-426f-b660-0f799d901608 service glance] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:47.525332 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc831d8-05ce-426f-b660-0f799d901608 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:47.569308 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc831d8-05ce-426f-b660-0f799d901608 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:47.585500 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc831d8-05ce-426f-b660-0f799d901608 service glance] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:47.607021 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1199/2360] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:33:47 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:33:47.804802 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600af6c4-23ef-45f7-bd25-07692e378f06 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:47.804802 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-600af6c4-23ef-45f7-bd25-07692e378f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:47.985821 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600af6c4-23ef-45f7-bd25-07692e378f06 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:47.998010 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd8232-5ab1-4a74-aff0-9bbb10f01fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:47.999043 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd8232-5ab1-4a74-aff0-9bbb10f01fbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:47.999330 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd8232-5ab1-4a74-aff0-9bbb10f01fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:47.999606 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd8232-5ab1-4a74-aff0-9bbb10f01fbe admin admin] PATH_INFO: `/v3/projects/b49d6e0d43eb4c1bb85fb49bc7a0d77b` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:48.001189 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffd8232-5ab1-4a74-aff0-9bbb10f01fbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=b49d6e0d43eb4c1bb85fb49bc7a0d77b)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:48.003876 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffd8232-5ab1-4a74-aff0-9bbb10f01fbe admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:48.159892 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-fffd8232-5ab1-4a74-aff0-9bbb10f01fbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b49d6e0d43eb4c1bb85fb49bc7a0d77b', 'request_id': 'req-fffd8232-5ab1-4a74-aff0-9bbb10f01fbe'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:48.171627 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-fffd8232-5ab1-4a74-aff0-9bbb10f01fbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b49d6e0d43eb4c1bb85fb49bc7a0d77b', 'request_id': 'req-fffd8232-5ab1-4a74-aff0-9bbb10f01fbe'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:48.179665 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:48.198438 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-fffd8232-5ab1-4a74-aff0-9bbb10f01fbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b49d6e0d43eb4c1bb85fb49bc7a0d77b', 'request_id': 'req-fffd8232-5ab1-4a74-aff0-9bbb10f01fbe'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:48.251547 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-fffd8232-5ab1-4a74-aff0-9bbb10f01fbe admin admin] The token cache is being invalidate because project b49d6e0d43eb4c1bb85fb49bc7a0d77b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:33:48.252091 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-fffd8232-5ab1-4a74-aff0-9bbb10f01fbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:48.253796 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-fffd8232-5ab1-4a74-aff0-9bbb10f01fbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:48.261727 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1162/2361] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:33:47 2022] DELETE /identity/v3/projects/b49d6e0d43eb4c1bb85fb49bc7a0d77b => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:48.264580 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc831d8-05ce-426f-b660-0f799d901608 service glance] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:48.290305 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc831d8-05ce-426f-b660-0f799d901608 service glance] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:48.462489 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddff08e-e1a9-41bb-8b3a-ff8ef47af86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:48.464368 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddff08e-e1a9-41bb-8b3a-ff8ef47af86f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:48.465060 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddff08e-e1a9-41bb-8b3a-ff8ef47af86f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:48.465613 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddff08e-e1a9-41bb-8b3a-ff8ef47af86f admin admin] PATH_INFO: `/v3/users/233b67c192d04043b2fc4c0ceeb07776` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:48.468337 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddff08e-e1a9-41bb-8b3a-ff8ef47af86f admin admin] RBAC: Authorizing `identity:delete_user(user_id=233b67c192d04043b2fc4c0ceeb07776)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:48.472396 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddff08e-e1a9-41bb-8b3a-ff8ef47af86f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:48.666744 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-cddff08e-e1a9-41bb-8b3a-ff8ef47af86f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '233b67c192d04043b2fc4c0ceeb07776'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:48.702930 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-cddff08e-e1a9-41bb-8b3a-ff8ef47af86f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '233b67c192d04043b2fc4c0ceeb07776'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:48.704613 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-cddff08e-e1a9-41bb-8b3a-ff8ef47af86f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '233b67c192d04043b2fc4c0ceeb07776'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:48.706409 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-cddff08e-e1a9-41bb-8b3a-ff8ef47af86f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '233b67c192d04043b2fc4c0ceeb07776'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:48.723088 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-cddff08e-e1a9-41bb-8b3a-ff8ef47af86f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '233b67c192d04043b2fc4c0ceeb07776'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:48.770071 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1200/2362] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:33:48 2022] DELETE /identity/v3/users/233b67c192d04043b2fc4c0ceeb07776 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:49.481546 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd8232-5ab1-4a74-aff0-9bbb10f01fbe admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:49.484175 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fffd8232-5ab1-4a74-aff0-9bbb10f01fbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:49.510448 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd8232-5ab1-4a74-aff0-9bbb10f01fbe admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:49.604739 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93e4fde-a697-4c3c-8efa-a0f8be2db3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:49.605845 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93e4fde-a697-4c3c-8efa-a0f8be2db3b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:49.606313 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93e4fde-a697-4c3c-8efa-a0f8be2db3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:49.606793 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93e4fde-a697-4c3c-8efa-a0f8be2db3b2 admin admin] PATH_INFO: `/v3/projects/b62ca666f8794ae6bc4b5b7601cbde4b` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:49.608387 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93e4fde-a697-4c3c-8efa-a0f8be2db3b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b62ca666f8794ae6bc4b5b7601cbde4b)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:49.610862 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93e4fde-a697-4c3c-8efa-a0f8be2db3b2 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:49.675716 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a93e4fde-a697-4c3c-8efa-a0f8be2db3b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b62ca666f8794ae6bc4b5b7601cbde4b', 'request_id': 'req-a93e4fde-a697-4c3c-8efa-a0f8be2db3b2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:49.688143 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a93e4fde-a697-4c3c-8efa-a0f8be2db3b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b62ca666f8794ae6bc4b5b7601cbde4b', 'request_id': 'req-a93e4fde-a697-4c3c-8efa-a0f8be2db3b2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:49.696251 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a93e4fde-a697-4c3c-8efa-a0f8be2db3b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b62ca666f8794ae6bc4b5b7601cbde4b', 'request_id': 'req-a93e4fde-a697-4c3c-8efa-a0f8be2db3b2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:49.720461 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a93e4fde-a697-4c3c-8efa-a0f8be2db3b2 admin admin] The token cache is being invalidate because project b62ca666f8794ae6bc4b5b7601cbde4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:33:49.720980 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a93e4fde-a697-4c3c-8efa-a0f8be2db3b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:49.721826 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a93e4fde-a697-4c3c-8efa-a0f8be2db3b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:49.725188 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1163/2363] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:33:49 2022] DELETE /identity/v3/projects/b62ca666f8794ae6bc4b5b7601cbde4b => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:49.731228 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddff08e-e1a9-41bb-8b3a-ff8ef47af86f admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:49.733261 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-cddff08e-e1a9-41bb-8b3a-ff8ef47af86f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:49.754726 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddff08e-e1a9-41bb-8b3a-ff8ef47af86f admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:49.884758 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158166a6-c913-44be-9d26-75e52a6226a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:49.886253 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158166a6-c913-44be-9d26-75e52a6226a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:49.886746 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158166a6-c913-44be-9d26-75e52a6226a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:49.887202 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158166a6-c913-44be-9d26-75e52a6226a2 admin admin] PATH_INFO: `/v3/users/451b1b59d40841f9b348db16d13c407d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:49.889469 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158166a6-c913-44be-9d26-75e52a6226a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=451b1b59d40841f9b348db16d13c407d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:49.893066 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158166a6-c913-44be-9d26-75e52a6226a2 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:49.897756 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:49.981901 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-158166a6-c913-44be-9d26-75e52a6226a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '451b1b59d40841f9b348db16d13c407d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:50.006668 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-158166a6-c913-44be-9d26-75e52a6226a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '451b1b59d40841f9b348db16d13c407d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:50.008080 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-158166a6-c913-44be-9d26-75e52a6226a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '451b1b59d40841f9b348db16d13c407d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:50.009139 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-158166a6-c913-44be-9d26-75e52a6226a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '451b1b59d40841f9b348db16d13c407d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:50.021881 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-158166a6-c913-44be-9d26-75e52a6226a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '451b1b59d40841f9b348db16d13c407d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:50.046987 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1201/2364] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:33:49 2022] DELETE /identity/v3/users/451b1b59d40841f9b348db16d13c407d => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:50.477311 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93e4fde-a697-4c3c-8efa-a0f8be2db3b2 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:50.479547 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a93e4fde-a697-4c3c-8efa-a0f8be2db3b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:50.516401 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93e4fde-a697-4c3c-8efa-a0f8be2db3b2 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:50.682119 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c029d10-bf53-4803-b82a-e9ea4ac085df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:50.687957 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c029d10-bf53-4803-b82a-e9ea4ac085df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:50.688607 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c029d10-bf53-4803-b82a-e9ea4ac085df admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:50.689068 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c029d10-bf53-4803-b82a-e9ea4ac085df admin admin] PATH_INFO: `/v3/projects/fa935717fb134af4b7a4d0fe71940d20` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:50.691541 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c029d10-bf53-4803-b82a-e9ea4ac085df admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa935717fb134af4b7a4d0fe71940d20)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:50.696771 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c029d10-bf53-4803-b82a-e9ea4ac085df admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:50.755839 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:50.806684 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7c029d10-bf53-4803-b82a-e9ea4ac085df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa935717fb134af4b7a4d0fe71940d20', 'request_id': 'req-7c029d10-bf53-4803-b82a-e9ea4ac085df'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:50.821126 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7c029d10-bf53-4803-b82a-e9ea4ac085df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa935717fb134af4b7a4d0fe71940d20', 'request_id': 'req-7c029d10-bf53-4803-b82a-e9ea4ac085df'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:50.845086 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7c029d10-bf53-4803-b82a-e9ea4ac085df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa935717fb134af4b7a4d0fe71940d20', 'request_id': 'req-7c029d10-bf53-4803-b82a-e9ea4ac085df'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:50.868626 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7c029d10-bf53-4803-b82a-e9ea4ac085df admin admin] The token cache is being invalidate because project fa935717fb134af4b7a4d0fe71940d20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:33:50.869082 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7c029d10-bf53-4803-b82a-e9ea4ac085df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:50.869703 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7c029d10-bf53-4803-b82a-e9ea4ac085df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:50.871301 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1164/2365] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:33:50 2022] DELETE /identity/v3/projects/fa935717fb134af4b7a4d0fe71940d20 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:50.881204 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32efbd4d-dbb4-483b-b5dd-473e15b06182 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:50.882086 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32efbd4d-dbb4-483b-b5dd-473e15b06182 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:50.882240 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32efbd4d-dbb4-483b-b5dd-473e15b06182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:50.918442 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-32efbd4d-dbb4-483b-b5dd-473e15b06182 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:50.921097 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-32efbd4d-dbb4-483b-b5dd-473e15b06182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:51.045866 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1202/2366] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:33:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:33:51.059061 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920cb46-7fd8-41dc-a3c4-684198fad69d None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:51.059290 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920cb46-7fd8-41dc-a3c4-684198fad69d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:51.059526 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920cb46-7fd8-41dc-a3c4-684198fad69d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:51.090030 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-9920cb46-7fd8-41dc-a3c4-684198fad69d None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:51.092220 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9920cb46-7fd8-41dc-a3c4-684198fad69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:51.106720 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1165/2367] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:33:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:33:51.113898 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32efbd4d-dbb4-483b-b5dd-473e15b06182 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:51.136948 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32efbd4d-dbb4-483b-b5dd-473e15b06182 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:51.146321 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bb8d4f-71e0-42db-a40e-81df42d665d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:51.147088 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb8d4f-71e0-42db-a40e-81df42d665d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:51.147382 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb8d4f-71e0-42db-a40e-81df42d665d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:51.147678 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb8d4f-71e0-42db-a40e-81df42d665d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:51.148504 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bb8d4f-71e0-42db-a40e-81df42d665d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:33:51.148993 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bb8d4f-71e0-42db-a40e-81df42d665d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:51.150930 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bb8d4f-71e0-42db-a40e-81df42d665d7 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:51.173383 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1203/2368] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:33:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:33:51.179689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920cb46-7fd8-41dc-a3c4-684198fad69d None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:51.204976 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920cb46-7fd8-41dc-a3c4-684198fad69d None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:51.212875 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e62d8-902e-4887-99c1-7be63ea555ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:51.214215 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e62d8-902e-4887-99c1-7be63ea555ea admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:51.214630 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e62d8-902e-4887-99c1-7be63ea555ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:51.214630 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e62d8-902e-4887-99c1-7be63ea555ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:51.215682 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e62d8-902e-4887-99c1-7be63ea555ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:51.219079 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e62d8-902e-4887-99c1-7be63ea555ea admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:51.280357 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1166/2369] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:33:51 2022] POST /identity/v3/projects => generated 383 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:33:51.283089 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bb8d4f-71e0-42db-a40e-81df42d665d7 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:51.302103 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bb8d4f-71e0-42db-a40e-81df42d665d7 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:51.437750 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8feab3-6198-4acf-926e-fb42478a2332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:51.438997 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8feab3-6198-4acf-926e-fb42478a2332 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:51.439529 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8feab3-6198-4acf-926e-fb42478a2332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:51.440016 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8feab3-6198-4acf-926e-fb42478a2332 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:51.441630 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8feab3-6198-4acf-926e-fb42478a2332 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:51.446496 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8feab3-6198-4acf-926e-fb42478a2332 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:51.464701 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:51.484592 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1204/2370] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:33:51 2022] POST /identity/v3/users => generated 292 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:33:51.489164 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e62d8-902e-4887-99c1-7be63ea555ea admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:51.513373 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e62d8-902e-4887-99c1-7be63ea555ea admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:51.519981 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec46dc41-a2ec-4db9-89ec-43032614d1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:51.521090 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46dc41-a2ec-4db9-89ec-43032614d1dd admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:51.521367 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46dc41-a2ec-4db9-89ec-43032614d1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:51.521889 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46dc41-a2ec-4db9-89ec-43032614d1dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:51.522699 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec46dc41-a2ec-4db9-89ec-43032614d1dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:51.524724 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec46dc41-a2ec-4db9-89ec-43032614d1dd admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:51.544159 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1167/2371] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:33:51.551681 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8feab3-6198-4acf-926e-fb42478a2332 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:51.570511 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8feab3-6198-4acf-926e-fb42478a2332 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:51.578030 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f80cc04-d4f1-4473-b3d5-14d5f548d21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:51.579232 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f80cc04-d4f1-4473-b3d5-14d5f548d21e admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:51.579686 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f80cc04-d4f1-4473-b3d5-14d5f548d21e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:51.580190 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f80cc04-d4f1-4473-b3d5-14d5f548d21e admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:51.581418 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f80cc04-d4f1-4473-b3d5-14d5f548d21e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:51.584792 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f80cc04-d4f1-4473-b3d5-14d5f548d21e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:51.614709 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1205/2372] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:33:51.620172 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec46dc41-a2ec-4db9-89ec-43032614d1dd admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:51.640833 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec46dc41-a2ec-4db9-89ec-43032614d1dd admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:51.651054 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc02b75-9b84-46d6-8c94-931212874c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:51.652027 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc02b75-9b84-46d6-8c94-931212874c40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:51.652315 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc02b75-9b84-46d6-8c94-931212874c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:51.652708 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc02b75-9b84-46d6-8c94-931212874c40 admin admin] PATH_INFO: `/v3/projects/1c6d76c7475d415d99a8a9a57729d05e/users/a8f7f7caa3424b70b2206be05fe36b6c/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:51.676000 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc02b75-9b84-46d6-8c94-931212874c40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c6d76c7475d415d99a8a9a57729d05e, user_id=a8f7f7caa3424b70b2206be05fe36b6c, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:51.682400 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc02b75-9b84-46d6-8c94-931212874c40 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:51.705902 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1168/2373] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:33:51 2022] PUT /identity/v3/projects/1c6d76c7475d415d99a8a9a57729d05e/users/a8f7f7caa3424b70b2206be05fe36b6c/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:51.734024 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c1ec9-1d4e-4526-acdd-33eadadf464a None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:51.734992 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c1ec9-1d4e-4526-acdd-33eadadf464a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:51.736647 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c1ec9-1d4e-4526-acdd-33eadadf464a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:51.812678 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-de6c1ec9-1d4e-4526-acdd-33eadadf464a None None] MFA Rules not processed for user `a8f7f7caa3424b70b2206be05fe36b6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:51.815946 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-de6c1ec9-1d4e-4526-acdd-33eadadf464a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:51.989845 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1206/2374] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:33:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:33:52.011519 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc02b75-9b84-46d6-8c94-931212874c40 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:52.023270 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-acc02b75-9b84-46d6-8c94-931212874c40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:52.119309 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce19a3-26b4-4b80-bd67-c27e5e1072ab None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:52.120338 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce19a3-26b4-4b80-bd67-c27e5e1072ab None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:52.120856 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce19a3-26b4-4b80-bd67-c27e5e1072ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:52.144732 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-f2ce19a3-26b4-4b80-bd67-c27e5e1072ab None None] MFA Rules not processed for user `09c9fa7dccb8491c9ad3b43967345ea2`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:52.146739 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f2ce19a3-26b4-4b80-bd67-c27e5e1072ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:52.186993 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc02b75-9b84-46d6-8c94-931212874c40 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:52.196297 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd95a7d-c3f1-426e-86b7-35ecafba64cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:52.197547 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd95a7d-c3f1-426e-86b7-35ecafba64cb service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:52.198135 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd95a7d-c3f1-426e-86b7-35ecafba64cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:52.198708 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd95a7d-c3f1-426e-86b7-35ecafba64cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:52.232995 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd95a7d-c3f1-426e-86b7-35ecafba64cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:52.236060 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd95a7d-c3f1-426e-86b7-35ecafba64cb service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:52.270326 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1169/2375] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:33:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:33:52.314777 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1207/2376] 10.222.0.87 () {58 vars in 1009 bytes} [Tue Jun 21 18:33:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:33:52.336580 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd95a7d-c3f1-426e-86b7-35ecafba64cb service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:52.338873 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9bd95a7d-c3f1-426e-86b7-35ecafba64cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:52.425328 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:52.455564 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd95a7d-c3f1-426e-86b7-35ecafba64cb service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:52.466796 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d2e95e-4903-4afc-b592-245a37fab429 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '515b3571045e4fe29565307c8ecc10b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:52.467924 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d2e95e-4903-4afc-b592-245a37fab429 service swift] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:52.468520 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d2e95e-4903-4afc-b592-245a37fab429 service swift] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:52.468992 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d2e95e-4903-4afc-b592-245a37fab429 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:52.494232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d2e95e-4903-4afc-b592-245a37fab429 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:52.498807 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d2e95e-4903-4afc-b592-245a37fab429 service swift] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:52.517497 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1170/2377] 10.222.0.87 () {60 vars in 1384 bytes} [Tue Jun 21 18:33:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 18:33:53.665775 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad33336-0802-4210-b061-30093f295dbd None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:53.667965 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad33336-0802-4210-b061-30093f295dbd None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:53.667965 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad33336-0802-4210-b061-30093f295dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:53.708029 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-5ad33336-0802-4210-b061-30093f295dbd None None] MFA Rules not processed for user `09c9fa7dccb8491c9ad3b43967345ea2`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:53.710618 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5ad33336-0802-4210-b061-30093f295dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:53.724191 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1208/2378] 10.222.0.87 () {58 vars in 967 bytes} [Tue Jun 21 18:33:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:33:53.745361 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d2e95e-4903-4afc-b592-245a37fab429 service swift] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:53.773068 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d2e95e-4903-4afc-b592-245a37fab429 service swift] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:53.785500 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cde345-f17c-40a1-9e2c-0cab145d9e29 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '515b3571045e4fe29565307c8ecc10b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:53.787442 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cde345-f17c-40a1-9e2c-0cab145d9e29 service swift] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:53.788356 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cde345-f17c-40a1-9e2c-0cab145d9e29 service swift] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:53.788860 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cde345-f17c-40a1-9e2c-0cab145d9e29 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:53.804989 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cde345-f17c-40a1-9e2c-0cab145d9e29 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:53.808123 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cde345-f17c-40a1-9e2c-0cab145d9e29 service swift] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:53.827154 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1171/2379] 10.222.0.87 () {60 vars in 1384 bytes} [Tue Jun 21 18:33:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 21 18:33:54.187749 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e9dee3-a2a4-423d-a3f7-ac66ba5ef9c0 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:54.188293 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e9dee3-a2a4-423d-a3f7-ac66ba5ef9c0 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:54.188778 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e9dee3-a2a4-423d-a3f7-ac66ba5ef9c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:54.221295 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-a2e9dee3-a2a4-423d-a3f7-ac66ba5ef9c0 None None] MFA Rules not processed for user `09c9fa7dccb8491c9ad3b43967345ea2`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:54.224345 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a2e9dee3-a2a4-423d-a3f7-ac66ba5ef9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:54.245533 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1209/2380] 10.222.0.87 () {58 vars in 1009 bytes} [Tue Jun 21 18:33:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:33:54.263891 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cde345-f17c-40a1-9e2c-0cab145d9e29 service swift] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:54.290151 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cde345-f17c-40a1-9e2c-0cab145d9e29 service swift] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:54.298769 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56096026-6d70-401e-ab50-c4213fa0d5fa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '515b3571045e4fe29565307c8ecc10b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:54.299567 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56096026-6d70-401e-ab50-c4213fa0d5fa service swift] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:54.299972 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56096026-6d70-401e-ab50-c4213fa0d5fa service swift] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:54.300365 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56096026-6d70-401e-ab50-c4213fa0d5fa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:54.326607 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56096026-6d70-401e-ab50-c4213fa0d5fa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:54.329589 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56096026-6d70-401e-ab50-c4213fa0d5fa service swift] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:54.345067 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1172/2381] 10.222.0.87 () {60 vars in 1384 bytes} [Tue Jun 21 18:33:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 18:33:54.896039 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4119f3b-ec67-4e03-b947-2685fdbad738 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:54.897243 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4119f3b-ec67-4e03-b947-2685fdbad738 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:54.898023 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4119f3b-ec67-4e03-b947-2685fdbad738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:54.933344 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-a4119f3b-ec67-4e03-b947-2685fdbad738 None None] MFA Rules not processed for user `09c9fa7dccb8491c9ad3b43967345ea2`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:54.936565 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a4119f3b-ec67-4e03-b947-2685fdbad738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:54.950055 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1210/2382] 10.222.0.87 () {58 vars in 967 bytes} [Tue Jun 21 18:33:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:33:54.987786 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56096026-6d70-401e-ab50-c4213fa0d5fa service swift] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:55.002245 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56096026-6d70-401e-ab50-c4213fa0d5fa service swift] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:55.008616 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1308afa0-7233-42b5-89bc-c809491af002 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '515b3571045e4fe29565307c8ecc10b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:55.009306 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1308afa0-7233-42b5-89bc-c809491af002 service swift] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:55.009597 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1308afa0-7233-42b5-89bc-c809491af002 service swift] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:55.009926 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1308afa0-7233-42b5-89bc-c809491af002 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:55.023393 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1308afa0-7233-42b5-89bc-c809491af002 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:55.026273 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1308afa0-7233-42b5-89bc-c809491af002 service swift] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:55.039950 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1173/2383] 10.222.0.87 () {60 vars in 1384 bytes} [Tue Jun 21 18:33:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 21 18:34:02.009338 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4119f3b-ec67-4e03-b947-2685fdbad738 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:02.130595 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1308afa0-7233-42b5-89bc-c809491af002 service swift] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:02.133295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1308afa0-7233-42b5-89bc-c809491af002 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:02.136855 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4119f3b-ec67-4e03-b947-2685fdbad738 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:02.145824 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f240b-46dc-4f1f-98f1-52b53e938496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:02.147404 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f240b-46dc-4f1f-98f1-52b53e938496 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:02.148010 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f240b-46dc-4f1f-98f1-52b53e938496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:02.148920 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f240b-46dc-4f1f-98f1-52b53e938496 admin admin] PATH_INFO: `/v3/users/a8f7f7caa3424b70b2206be05fe36b6c` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:02.153520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4f240b-46dc-4f1f-98f1-52b53e938496 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8f7f7caa3424b70b2206be05fe36b6c)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:02.157974 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4f240b-46dc-4f1f-98f1-52b53e938496 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:02.160503 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:34:02.161199 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1308afa0-7233-42b5-89bc-c809491af002 service swift] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:02.171764 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9828996a-90ea-4a65-8b0c-91c8ebe39e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:02.173854 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828996a-90ea-4a65-8b0c-91c8ebe39e92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:02.174058 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828996a-90ea-4a65-8b0c-91c8ebe39e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:02.174371 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828996a-90ea-4a65-8b0c-91c8ebe39e92 admin admin] PATH_INFO: `/v3/users/2bbfa51f793c4d2fb53ce386f4d33192` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:02.176649 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9828996a-90ea-4a65-8b0c-91c8ebe39e92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bbfa51f793c4d2fb53ce386f4d33192)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:02.180407 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9828996a-90ea-4a65-8b0c-91c8ebe39e92 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:02.257317 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-bf4f240b-46dc-4f1f-98f1-52b53e938496 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8f7f7caa3424b70b2206be05fe36b6c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:02.287921 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-bf4f240b-46dc-4f1f-98f1-52b53e938496 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8f7f7caa3424b70b2206be05fe36b6c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:02.289390 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-bf4f240b-46dc-4f1f-98f1-52b53e938496 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8f7f7caa3424b70b2206be05fe36b6c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:02.291492 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-bf4f240b-46dc-4f1f-98f1-52b53e938496 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8f7f7caa3424b70b2206be05fe36b6c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:02.299647 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9828996a-90ea-4a65-8b0c-91c8ebe39e92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bbfa51f793c4d2fb53ce386f4d33192'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:02.321805 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9828996a-90ea-4a65-8b0c-91c8ebe39e92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bbfa51f793c4d2fb53ce386f4d33192'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:02.323251 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9828996a-90ea-4a65-8b0c-91c8ebe39e92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bbfa51f793c4d2fb53ce386f4d33192'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:02.324306 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9828996a-90ea-4a65-8b0c-91c8ebe39e92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bbfa51f793c4d2fb53ce386f4d33192'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:02.331395 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-bf4f240b-46dc-4f1f-98f1-52b53e938496 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8f7f7caa3424b70b2206be05fe36b6c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:02.362582 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9828996a-90ea-4a65-8b0c-91c8ebe39e92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bbfa51f793c4d2fb53ce386f4d33192'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:02.365559 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1211/2384] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:34:02 2022] DELETE /identity/v3/users/a8f7f7caa3424b70b2206be05fe36b6c => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:34:02.374098 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca82f6-87f5-458f-92a0-254aca4cec67 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:02.374098 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca82f6-87f5-458f-92a0-254aca4cec67 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:02.374098 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca82f6-87f5-458f-92a0-254aca4cec67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:02.403794 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1174/2385] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:34:02 2022] DELETE /identity/v3/users/2bbfa51f793c4d2fb53ce386f4d33192 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:34:02.411239 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-d4ca82f6-87f5-458f-92a0-254aca4cec67 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:34:02.412100 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2046ed16-312f-4d8e-a5c9-cd7a86bd0642 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:02.412641 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2046ed16-312f-4d8e-a5c9-cd7a86bd0642 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:02.413013 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d4ca82f6-87f5-458f-92a0-254aca4cec67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:02.413121 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2046ed16-312f-4d8e-a5c9-cd7a86bd0642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:02.458758 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-2046ed16-312f-4d8e-a5c9-cd7a86bd0642 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:34:02.459936 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2046ed16-312f-4d8e-a5c9-cd7a86bd0642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:02.572024 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1212/2386] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:34:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:34:02.585712 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ca82f6-87f5-458f-92a0-254aca4cec67 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:02.588786 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d4ca82f6-87f5-458f-92a0-254aca4cec67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:02.589275 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1175/2387] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:34:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:34:02.611436 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2046ed16-312f-4d8e-a5c9-cd7a86bd0642 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:02.615678 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2046ed16-312f-4d8e-a5c9-cd7a86bd0642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:02.645811 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:34:02.738461 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ca82f6-87f5-458f-92a0-254aca4cec67 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:02.744930 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe4876b-6807-44d9-935e-5578dd35878f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:02.746206 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe4876b-6807-44d9-935e-5578dd35878f service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:02.746769 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe4876b-6807-44d9-935e-5578dd35878f service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:02.747358 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe4876b-6807-44d9-935e-5578dd35878f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:02.759726 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2046ed16-312f-4d8e-a5c9-cd7a86bd0642 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:02.765097 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe4876b-6807-44d9-935e-5578dd35878f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:02.766308 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d78bc1-851f-4c16-ad12-2fba2a629175 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:02.767056 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d78bc1-851f-4c16-ad12-2fba2a629175 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:02.767254 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe4876b-6807-44d9-935e-5578dd35878f service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:02.767416 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d78bc1-851f-4c16-ad12-2fba2a629175 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:02.767609 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d78bc1-851f-4c16-ad12-2fba2a629175 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:02.783198 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1213/2388] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:34:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:34:02.789861 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d78bc1-851f-4c16-ad12-2fba2a629175 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:02.792845 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d78bc1-851f-4c16-ad12-2fba2a629175 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:02.818377 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1176/2389] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:34:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:34:03.171619 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe4876b-6807-44d9-935e-5578dd35878f service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:03.174624 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4fe4876b-6807-44d9-935e-5578dd35878f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:03.212490 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe4876b-6807-44d9-935e-5578dd35878f service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:03.223562 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d288e-8090-4da9-8306-053fe70f253b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:03.227253 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d288e-8090-4da9-8306-053fe70f253b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:03.227632 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d288e-8090-4da9-8306-053fe70f253b admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:03.228327 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d288e-8090-4da9-8306-053fe70f253b admin admin] PATH_INFO: `/v3/projects/b5569463610e44dc82d15553cbbcb02c` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:03.231886 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167d288e-8090-4da9-8306-053fe70f253b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5569463610e44dc82d15553cbbcb02c)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:03.236233 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167d288e-8090-4da9-8306-053fe70f253b admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:03.274514 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:34:03.295993 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d78bc1-851f-4c16-ad12-2fba2a629175 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:03.299086 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b0d78bc1-851f-4c16-ad12-2fba2a629175 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:03.326788 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d78bc1-851f-4c16-ad12-2fba2a629175 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:03.366369 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-167d288e-8090-4da9-8306-053fe70f253b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5569463610e44dc82d15553cbbcb02c', 'request_id': 'req-167d288e-8090-4da9-8306-053fe70f253b'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:03.380577 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-167d288e-8090-4da9-8306-053fe70f253b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5569463610e44dc82d15553cbbcb02c', 'request_id': 'req-167d288e-8090-4da9-8306-053fe70f253b'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:03.393577 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-167d288e-8090-4da9-8306-053fe70f253b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5569463610e44dc82d15553cbbcb02c', 'request_id': 'req-167d288e-8090-4da9-8306-053fe70f253b'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:03.418874 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-167d288e-8090-4da9-8306-053fe70f253b admin admin] The token cache is being invalidate because project b5569463610e44dc82d15553cbbcb02c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:34:03.419424 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-167d288e-8090-4da9-8306-053fe70f253b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:03.420698 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-167d288e-8090-4da9-8306-053fe70f253b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:03.424171 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1214/2390] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:34:03 2022] DELETE /identity/v3/projects/b5569463610e44dc82d15553cbbcb02c => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:34:03.453866 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8922a7b4-db70-4c71-a375-3570fe018323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:03.455239 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8922a7b4-db70-4c71-a375-3570fe018323 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:03.455661 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8922a7b4-db70-4c71-a375-3570fe018323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:03.456079 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8922a7b4-db70-4c71-a375-3570fe018323 admin admin] PATH_INFO: `/v3/projects/1c6d76c7475d415d99a8a9a57729d05e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:03.458340 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8922a7b4-db70-4c71-a375-3570fe018323 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c6d76c7475d415d99a8a9a57729d05e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:03.462107 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8922a7b4-db70-4c71-a375-3570fe018323 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:03.608593 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8922a7b4-db70-4c71-a375-3570fe018323 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c6d76c7475d415d99a8a9a57729d05e', 'request_id': 'req-8922a7b4-db70-4c71-a375-3570fe018323'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:03.629068 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8922a7b4-db70-4c71-a375-3570fe018323 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c6d76c7475d415d99a8a9a57729d05e', 'request_id': 'req-8922a7b4-db70-4c71-a375-3570fe018323'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:03.642469 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8922a7b4-db70-4c71-a375-3570fe018323 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c6d76c7475d415d99a8a9a57729d05e', 'request_id': 'req-8922a7b4-db70-4c71-a375-3570fe018323'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:03.677592 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8922a7b4-db70-4c71-a375-3570fe018323 admin admin] The token cache is being invalidate because project 1c6d76c7475d415d99a8a9a57729d05e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:34:03.678146 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8922a7b4-db70-4c71-a375-3570fe018323 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:03.681117 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8922a7b4-db70-4c71-a375-3570fe018323 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:03.684051 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1177/2391] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:34:03 2022] DELETE /identity/v3/projects/1c6d76c7475d415d99a8a9a57729d05e => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:34:03.694119 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58787ba3-e8e9-42f9-a159-5c25548e6769 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:03.694706 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58787ba3-e8e9-42f9-a159-5c25548e6769 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:03.695132 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58787ba3-e8e9-42f9-a159-5c25548e6769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:03.735793 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-58787ba3-e8e9-42f9-a159-5c25548e6769 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:34:03.737505 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-58787ba3-e8e9-42f9-a159-5c25548e6769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:03.909509 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1215/2392] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:34:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:34:03.923112 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6264a1b-9ed6-47bb-b387-619570035897 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:03.923655 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6264a1b-9ed6-47bb-b387-619570035897 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:03.924161 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6264a1b-9ed6-47bb-b387-619570035897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:03.960021 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b6264a1b-9ed6-47bb-b387-619570035897 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:34:03.980559 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b6264a1b-9ed6-47bb-b387-619570035897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:04.017689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1178/2393] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:34:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:34:04.024055 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58787ba3-e8e9-42f9-a159-5c25548e6769 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:04.036738 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58787ba3-e8e9-42f9-a159-5c25548e6769 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:04.043318 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a8f52-5864-45c5-96e8-a3de9c72ea60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:04.044520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a8f52-5864-45c5-96e8-a3de9c72ea60 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:04.045008 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a8f52-5864-45c5-96e8-a3de9c72ea60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:04.045459 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a8f52-5864-45c5-96e8-a3de9c72ea60 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:04.046789 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3a8f52-5864-45c5-96e8-a3de9c72ea60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:34:04.047825 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3a8f52-5864-45c5-96e8-a3de9c72ea60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:04.051553 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3a8f52-5864-45c5-96e8-a3de9c72ea60 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:04.098545 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1216/2394] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:34:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:34:04.101862 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6264a1b-9ed6-47bb-b387-619570035897 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:04.118254 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6264a1b-9ed6-47bb-b387-619570035897 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:04.128913 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922dcd0a-75b0-4865-aa2b-234eb17f6aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:04.130777 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922dcd0a-75b0-4865-aa2b-234eb17f6aff admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:04.130777 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922dcd0a-75b0-4865-aa2b-234eb17f6aff admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:04.131382 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922dcd0a-75b0-4865-aa2b-234eb17f6aff admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:04.132313 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922dcd0a-75b0-4865-aa2b-234eb17f6aff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:04.135031 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922dcd0a-75b0-4865-aa2b-234eb17f6aff admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:04.185706 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1179/2395] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:34:04 2022] POST /identity/v3/projects => generated 375 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:34:04.189518 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a8f52-5864-45c5-96e8-a3de9c72ea60 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:04.204270 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a8f52-5864-45c5-96e8-a3de9c72ea60 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:04.318575 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f792e2-9b2a-44b8-9aec-55295bd1f483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:04.319694 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f792e2-9b2a-44b8-9aec-55295bd1f483 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:04.319978 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f792e2-9b2a-44b8-9aec-55295bd1f483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:04.320494 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f792e2-9b2a-44b8-9aec-55295bd1f483 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:04.321418 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f792e2-9b2a-44b8-9aec-55295bd1f483 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:04.323815 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f792e2-9b2a-44b8-9aec-55295bd1f483 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:04.369130 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1217/2396] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:34:04 2022] POST /identity/v3/users => generated 288 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:34:04.375338 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922dcd0a-75b0-4865-aa2b-234eb17f6aff admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:04.399494 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922dcd0a-75b0-4865-aa2b-234eb17f6aff admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:04.408713 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a53d4-c29e-4c75-9dee-f3fda13e5672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:04.409957 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a53d4-c29e-4c75-9dee-f3fda13e5672 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:04.410447 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a53d4-c29e-4c75-9dee-f3fda13e5672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:04.410932 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a53d4-c29e-4c75-9dee-f3fda13e5672 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:04.412637 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218a53d4-c29e-4c75-9dee-f3fda13e5672 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:04.415756 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218a53d4-c29e-4c75-9dee-f3fda13e5672 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:04.441511 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1180/2397] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:34:04 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:34:04.443675 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f792e2-9b2a-44b8-9aec-55295bd1f483 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:04.467989 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f792e2-9b2a-44b8-9aec-55295bd1f483 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:04.482105 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce0855-3ca6-4ff2-b753-3c0ac5bba0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:04.483442 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce0855-3ca6-4ff2-b753-3c0ac5bba0d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:04.483943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce0855-3ca6-4ff2-b753-3c0ac5bba0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:04.484400 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce0855-3ca6-4ff2-b753-3c0ac5bba0d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:04.485762 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ce0855-3ca6-4ff2-b753-3c0ac5bba0d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:04.489677 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ce0855-3ca6-4ff2-b753-3c0ac5bba0d6 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:04.512809 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1218/2398] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:34:04 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:34:04.519697 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a53d4-c29e-4c75-9dee-f3fda13e5672 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:04.530235 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:34:04.540938 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a53d4-c29e-4c75-9dee-f3fda13e5672 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:04.551352 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8398b5e3-2c45-45d7-ba6b-41c715714d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:04.553756 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8398b5e3-2c45-45d7-ba6b-41c715714d18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:04.554163 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8398b5e3-2c45-45d7-ba6b-41c715714d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:04.554542 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8398b5e3-2c45-45d7-ba6b-41c715714d18 admin admin] PATH_INFO: `/v3/projects/386114865dda4f1681c2fad550211fb7/users/db8a896128a746f4ad0544f55178f689/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:04.589219 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8398b5e3-2c45-45d7-ba6b-41c715714d18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=386114865dda4f1681c2fad550211fb7, user_id=db8a896128a746f4ad0544f55178f689, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:04.595287 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8398b5e3-2c45-45d7-ba6b-41c715714d18 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:04.643157 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1181/2399] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:34:04 2022] PUT /identity/v3/projects/386114865dda4f1681c2fad550211fb7/users/db8a896128a746f4ad0544f55178f689/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:34:04.656670 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d220e5b-4f05-4ec3-83b3-447e6bc637a7 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:04.657174 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d220e5b-4f05-4ec3-83b3-447e6bc637a7 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:04.657800 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d220e5b-4f05-4ec3-83b3-447e6bc637a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:04.692621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:34:04.704837 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-7d220e5b-4f05-4ec3-83b3-447e6bc637a7 None None] MFA Rules not processed for user `db8a896128a746f4ad0544f55178f689`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:34:04.707817 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7d220e5b-4f05-4ec3-83b3-447e6bc637a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:04.804791 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1219/2400] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:34:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:34:04.823144 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8398b5e3-2c45-45d7-ba6b-41c715714d18 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:04.826413 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8398b5e3-2c45-45d7-ba6b-41c715714d18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:04.958466 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8398b5e3-2c45-45d7-ba6b-41c715714d18 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:04.963779 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f3c34a-3194-4444-90fb-c1aa73582c01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:04.964613 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f3c34a-3194-4444-90fb-c1aa73582c01 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:04.964996 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f3c34a-3194-4444-90fb-c1aa73582c01 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:04.965377 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f3c34a-3194-4444-90fb-c1aa73582c01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:04.976515 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f3c34a-3194-4444-90fb-c1aa73582c01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:04.978217 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f3c34a-3194-4444-90fb-c1aa73582c01 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:04.989130 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1182/2401] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:34:04 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:34:09.421767 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f21d8-cc63-4707-bce9-824341ff61f1 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:09.422479 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f21d8-cc63-4707-bce9-824341ff61f1 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:09.422852 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f21d8-cc63-4707-bce9-824341ff61f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:09.450375 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-030f21d8-cc63-4707-bce9-824341ff61f1 None None] MFA Rules not processed for user `ec179985fc3b41d188ca69d9e03263d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:34:09.451886 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-030f21d8-cc63-4707-bce9-824341ff61f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:09.563376 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1220/2402] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:34:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:34:12.988170 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f3c34a-3194-4444-90fb-c1aa73582c01 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:12.993410 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-70f3c34a-3194-4444-90fb-c1aa73582c01 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:13.100589 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f3c34a-3194-4444-90fb-c1aa73582c01 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:13.107639 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c3b066-c67a-4a7d-9d79-a8c0f5da50bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:13.108778 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c3b066-c67a-4a7d-9d79-a8c0f5da50bf service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:13.109276 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c3b066-c67a-4a7d-9d79-a8c0f5da50bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:13.109829 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c3b066-c67a-4a7d-9d79-a8c0f5da50bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:13.113439 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c3c3b066-c67a-4a7d-9d79-a8c0f5da50bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:13.296466 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c3b066-c67a-4a7d-9d79-a8c0f5da50bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:13.298116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c3b066-c67a-4a7d-9d79-a8c0f5da50bf service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:13.319650 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1183/2403] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:34:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:34:15.804150 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030f21d8-cc63-4707-bce9-824341ff61f1 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:15.817071 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030f21d8-cc63-4707-bce9-824341ff61f1 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:15.822483 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd83427f-a7da-4d9c-b738-a9884c77d79b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:15.823364 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd83427f-a7da-4d9c-b738-a9884c77d79b service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:15.823492 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd83427f-a7da-4d9c-b738-a9884c77d79b service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:15.823715 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd83427f-a7da-4d9c-b738-a9884c77d79b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:15.826039 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-cd83427f-a7da-4d9c-b738-a9884c77d79b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:15.851022 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd83427f-a7da-4d9c-b738-a9884c77d79b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:15.853555 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd83427f-a7da-4d9c-b738-a9884c77d79b service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:15.868554 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1221/2404] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:34:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:34:23.310719 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:34:23.639092 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c3b066-c67a-4a7d-9d79-a8c0f5da50bf service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:23.786521 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c3b066-c67a-4a7d-9d79-a8c0f5da50bf service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:23.795047 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb9de38-447e-4e7a-870c-9266ed655307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:23.796701 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9de38-447e-4e7a-870c-9266ed655307 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:23.797272 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9de38-447e-4e7a-870c-9266ed655307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:23.797718 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9de38-447e-4e7a-870c-9266ed655307 admin admin] PATH_INFO: `/v3/users/db8a896128a746f4ad0544f55178f689` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:23.800110 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb9de38-447e-4e7a-870c-9266ed655307 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db8a896128a746f4ad0544f55178f689)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:23.803077 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb9de38-447e-4e7a-870c-9266ed655307 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:23.939361 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-cfb9de38-447e-4e7a-870c-9266ed655307 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db8a896128a746f4ad0544f55178f689'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:23.968255 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-cfb9de38-447e-4e7a-870c-9266ed655307 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db8a896128a746f4ad0544f55178f689'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:23.970343 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-cfb9de38-447e-4e7a-870c-9266ed655307 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db8a896128a746f4ad0544f55178f689'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:23.971263 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-cfb9de38-447e-4e7a-870c-9266ed655307 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db8a896128a746f4ad0544f55178f689'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:24.009395 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-cfb9de38-447e-4e7a-870c-9266ed655307 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db8a896128a746f4ad0544f55178f689'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:24.028397 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1184/2405] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:34:23 2022] DELETE /identity/v3/users/db8a896128a746f4ad0544f55178f689 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:34:24.035035 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f845ae-05c6-4030-92e2-e5fcf63ab63e None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:24.035873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f845ae-05c6-4030-92e2-e5fcf63ab63e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:24.035873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f845ae-05c6-4030-92e2-e5fcf63ab63e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:24.065931 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-06f845ae-05c6-4030-92e2-e5fcf63ab63e None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:34:24.067304 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-06f845ae-05c6-4030-92e2-e5fcf63ab63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:24.166341 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1222/2406] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:34:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:34:24.183499 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb9de38-447e-4e7a-870c-9266ed655307 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:24.187679 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-cfb9de38-447e-4e7a-870c-9266ed655307 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:24.288019 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb9de38-447e-4e7a-870c-9266ed655307 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:24.297337 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc3a352-b452-4227-bc2a-41b1c780cca2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:24.298650 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3a352-b452-4227-bc2a-41b1c780cca2 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:24.299195 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3a352-b452-4227-bc2a-41b1c780cca2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:24.300001 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3a352-b452-4227-bc2a-41b1c780cca2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:24.318349 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc3a352-b452-4227-bc2a-41b1c780cca2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:24.322677 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc3a352-b452-4227-bc2a-41b1c780cca2 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:24.342611 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1185/2407] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:34:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:34:25.050441 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f845ae-05c6-4030-92e2-e5fcf63ab63e None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:25.053980 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-06f845ae-05c6-4030-92e2-e5fcf63ab63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:25.080857 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f845ae-05c6-4030-92e2-e5fcf63ab63e None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:25.091828 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f182ac1c-c108-48b7-bb79-791f3060c442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:25.094085 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182ac1c-c108-48b7-bb79-791f3060c442 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:25.094831 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182ac1c-c108-48b7-bb79-791f3060c442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:25.095554 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182ac1c-c108-48b7-bb79-791f3060c442 admin admin] PATH_INFO: `/v3/projects/386114865dda4f1681c2fad550211fb7` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:25.099504 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f182ac1c-c108-48b7-bb79-791f3060c442 admin admin] RBAC: Authorizing `identity:delete_project(project_id=386114865dda4f1681c2fad550211fb7)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:25.103639 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f182ac1c-c108-48b7-bb79-791f3060c442 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:25.201610 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:34:25.242496 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-f182ac1c-c108-48b7-bb79-791f3060c442 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '386114865dda4f1681c2fad550211fb7', 'request_id': 'req-f182ac1c-c108-48b7-bb79-791f3060c442'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:25.256832 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-f182ac1c-c108-48b7-bb79-791f3060c442 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '386114865dda4f1681c2fad550211fb7', 'request_id': 'req-f182ac1c-c108-48b7-bb79-791f3060c442'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:25.277762 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-f182ac1c-c108-48b7-bb79-791f3060c442 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '386114865dda4f1681c2fad550211fb7', 'request_id': 'req-f182ac1c-c108-48b7-bb79-791f3060c442'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:25.304017 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-f182ac1c-c108-48b7-bb79-791f3060c442 admin admin] The token cache is being invalidate because project 386114865dda4f1681c2fad550211fb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:34:25.304304 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-f182ac1c-c108-48b7-bb79-791f3060c442 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:25.305116 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-f182ac1c-c108-48b7-bb79-791f3060c442 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:25.308171 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1223/2408] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:34:25 2022] DELETE /identity/v3/projects/386114865dda4f1681c2fad550211fb7 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:34:26.402768 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ae841-acfe-47bf-975f-74feb1e78bf8 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:26.403668 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ae841-acfe-47bf-975f-74feb1e78bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:26.403834 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ae841-acfe-47bf-975f-74feb1e78bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:26.430723 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-8e4ae841-acfe-47bf-975f-74feb1e78bf8 None None] MFA Rules not processed for user `144c9c31ed9b469aa200dcb16c89ab4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:34:26.432918 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8e4ae841-acfe-47bf-975f-74feb1e78bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:26.470068 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:34:26.547390 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1186/2409] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:34:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:34:27.160756 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f182ac1c-c108-48b7-bb79-791f3060c442 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:27.163172 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f182ac1c-c108-48b7-bb79-791f3060c442 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:27.280562 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f182ac1c-c108-48b7-bb79-791f3060c442 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:27.287603 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ed7f8d-775a-4a0c-9c06-feab1e724526 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:27.288343 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed7f8d-775a-4a0c-9c06-feab1e724526 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:27.288628 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed7f8d-775a-4a0c-9c06-feab1e724526 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:27.288991 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed7f8d-775a-4a0c-9c06-feab1e724526 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:27.292756 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-61ed7f8d-775a-4a0c-9c06-feab1e724526 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:27.309073 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ed7f8d-775a-4a0c-9c06-feab1e724526 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:27.311005 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ed7f8d-775a-4a0c-9c06-feab1e724526 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:27.328753 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1224/2410] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:34:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:34:27.837875 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ae841-acfe-47bf-975f-74feb1e78bf8 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:27.842368 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8e4ae841-acfe-47bf-975f-74feb1e78bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:28.013226 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ae841-acfe-47bf-975f-74feb1e78bf8 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:28.023134 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0ac4aa-85a9-4894-a81d-07b63816a112 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef1d49b36cdc4665b82487be97e9dcd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:28.024474 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ac4aa-85a9-4894-a81d-07b63816a112 service placement] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:28.024985 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ac4aa-85a9-4894-a81d-07b63816a112 service placement] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:28.025482 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ac4aa-85a9-4894-a81d-07b63816a112 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:28.028947 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9f0ac4aa-85a9-4894-a81d-07b63816a112 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:28.052569 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0ac4aa-85a9-4894-a81d-07b63816a112 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:28.055766 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0ac4aa-85a9-4894-a81d-07b63816a112 service placement] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:28.076120 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1187/2411] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:34:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:34:35.983994 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ed7f8d-775a-4a0c-9c06-feab1e724526 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:35.986080 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-61ed7f8d-775a-4a0c-9c06-feab1e724526 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:36.005979 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ed7f8d-775a-4a0c-9c06-feab1e724526 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:36.012130 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8efce6e-194e-4803-b2b5-4b08a7773f99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:36.012966 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efce6e-194e-4803-b2b5-4b08a7773f99 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:36.013478 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efce6e-194e-4803-b2b5-4b08a7773f99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:36.014003 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efce6e-194e-4803-b2b5-4b08a7773f99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:36.016700 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b8efce6e-194e-4803-b2b5-4b08a7773f99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:36.030434 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8efce6e-194e-4803-b2b5-4b08a7773f99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:36.032738 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8efce6e-194e-4803-b2b5-4b08a7773f99 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:36.048584 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1225/2412] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:34:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:34:38.067257 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=47 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:34:45.232416 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0ac4aa-85a9-4894-a81d-07b63816a112 service placement] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:45.248635 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0ac4aa-85a9-4894-a81d-07b63816a112 service placement] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:45.254777 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3408d4-4b61-4e0a-abcc-b21252ad199c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:45.255726 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3408d4-4b61-4e0a-abcc-b21252ad199c service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:45.256057 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3408d4-4b61-4e0a-abcc-b21252ad199c service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:45.256394 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3408d4-4b61-4e0a-abcc-b21252ad199c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:45.259280 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9a3408d4-4b61-4e0a-abcc-b21252ad199c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:45.275356 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3408d4-4b61-4e0a-abcc-b21252ad199c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:45.277109 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3408d4-4b61-4e0a-abcc-b21252ad199c service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:45.297420 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1188/2413] 10.222.0.87 () {60 vars in 1396 bytes} [Tue Jun 21 18:34:45 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3252 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:34:46.040730 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=46,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:34:48.252331 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8efce6e-194e-4803-b2b5-4b08a7773f99 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:48.275058 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8efce6e-194e-4803-b2b5-4b08a7773f99 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:48.294940 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cebedd-230a-4c1f-bf2c-1f23a94f4632 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:48.296656 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cebedd-230a-4c1f-bf2c-1f23a94f4632 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:48.296949 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cebedd-230a-4c1f-bf2c-1f23a94f4632 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:48.297259 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cebedd-230a-4c1f-bf2c-1f23a94f4632 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:48.299774 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b7cebedd-230a-4c1f-bf2c-1f23a94f4632 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:48.391795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cebedd-230a-4c1f-bf2c-1f23a94f4632 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:48.393533 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cebedd-230a-4c1f-bf2c-1f23a94f4632 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:48.404853 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1226/2414] 10.222.0.87 () {60 vars in 1396 bytes} [Tue Jun 21 18:34:48 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3244 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:34:55.283394 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=6 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:34:58.399846 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=28 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:35:05.738730 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3408d4-4b61-4e0a-abcc-b21252ad199c service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:05.740393 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9a3408d4-4b61-4e0a-abcc-b21252ad199c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:05.827792 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3408d4-4b61-4e0a-abcc-b21252ad199c service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:05.835553 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b7124f-465a-476d-8015-5d42f1954f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:05.836531 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b7124f-465a-476d-8015-5d42f1954f2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:05.836814 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b7124f-465a-476d-8015-5d42f1954f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:05.837095 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b7124f-465a-476d-8015-5d42f1954f2a admin admin] PATH_INFO: `/v3/users/fa5c0c04ba20442caebee478f2b52c0d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:05.839042 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b7124f-465a-476d-8015-5d42f1954f2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa5c0c04ba20442caebee478f2b52c0d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:05.842389 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b7124f-465a-476d-8015-5d42f1954f2a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:05.914839 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-89b7124f-465a-476d-8015-5d42f1954f2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa5c0c04ba20442caebee478f2b52c0d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:05.928900 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-89b7124f-465a-476d-8015-5d42f1954f2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa5c0c04ba20442caebee478f2b52c0d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:05.929936 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-89b7124f-465a-476d-8015-5d42f1954f2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa5c0c04ba20442caebee478f2b52c0d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:05.930829 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-89b7124f-465a-476d-8015-5d42f1954f2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa5c0c04ba20442caebee478f2b52c0d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:05.952305 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-89b7124f-465a-476d-8015-5d42f1954f2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa5c0c04ba20442caebee478f2b52c0d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:05.976301 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1189/2415] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:35:05 2022] DELETE /identity/v3/users/fa5c0c04ba20442caebee478f2b52c0d => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:35:05.982057 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615429ad-691f-43f8-a002-6622e9b73541 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:05.982768 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615429ad-691f-43f8-a002-6622e9b73541 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:05.982768 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615429ad-691f-43f8-a002-6622e9b73541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:06.005692 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-615429ad-691f-43f8-a002-6622e9b73541 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:35:06.007075 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-615429ad-691f-43f8-a002-6622e9b73541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:06.108568 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1227/2416] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:35:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:35:06.121101 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b7124f-465a-476d-8015-5d42f1954f2a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:06.124033 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-89b7124f-465a-476d-8015-5d42f1954f2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:06.211551 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b7124f-465a-476d-8015-5d42f1954f2a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:06.216434 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df06523-a591-4eed-b38c-af403fdd4501 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:06.217193 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df06523-a591-4eed-b38c-af403fdd4501 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:06.217521 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df06523-a591-4eed-b38c-af403fdd4501 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:06.217904 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df06523-a591-4eed-b38c-af403fdd4501 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:06.232800 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df06523-a591-4eed-b38c-af403fdd4501 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:06.234910 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df06523-a591-4eed-b38c-af403fdd4501 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:06.249496 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1190/2417] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:35:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:35:06.701763 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615429ad-691f-43f8-a002-6622e9b73541 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:06.704094 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-615429ad-691f-43f8-a002-6622e9b73541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:06.731212 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615429ad-691f-43f8-a002-6622e9b73541 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:06.741694 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7011b2c8-6a5d-404c-a6bd-839ae2191113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:06.743205 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7011b2c8-6a5d-404c-a6bd-839ae2191113 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:06.743745 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7011b2c8-6a5d-404c-a6bd-839ae2191113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:06.744213 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7011b2c8-6a5d-404c-a6bd-839ae2191113 admin admin] PATH_INFO: `/v3/projects/34c100a7ff2744348040ec3b057f3022` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:06.746761 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7011b2c8-6a5d-404c-a6bd-839ae2191113 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34c100a7ff2744348040ec3b057f3022)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:06.750651 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7011b2c8-6a5d-404c-a6bd-839ae2191113 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:06.829205 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7011b2c8-6a5d-404c-a6bd-839ae2191113 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34c100a7ff2744348040ec3b057f3022', 'request_id': 'req-7011b2c8-6a5d-404c-a6bd-839ae2191113'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:06.836865 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7011b2c8-6a5d-404c-a6bd-839ae2191113 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34c100a7ff2744348040ec3b057f3022', 'request_id': 'req-7011b2c8-6a5d-404c-a6bd-839ae2191113'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:06.849549 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7011b2c8-6a5d-404c-a6bd-839ae2191113 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34c100a7ff2744348040ec3b057f3022', 'request_id': 'req-7011b2c8-6a5d-404c-a6bd-839ae2191113'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:06.868987 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7011b2c8-6a5d-404c-a6bd-839ae2191113 admin admin] The token cache is being invalidate because project 34c100a7ff2744348040ec3b057f3022 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:35:06.869530 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7011b2c8-6a5d-404c-a6bd-839ae2191113 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:06.870190 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7011b2c8-6a5d-404c-a6bd-839ae2191113 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:06.872014 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1228/2418] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:35:06 2022] DELETE /identity/v3/projects/34c100a7ff2744348040ec3b057f3022 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:35:06.878561 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a921c-ac2e-4c43-a3e3-d50f3e7dadbf None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:06.879731 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a921c-ac2e-4c43-a3e3-d50f3e7dadbf None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:06.879731 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a921c-ac2e-4c43-a3e3-d50f3e7dadbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:06.909315 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-5d9a921c-ac2e-4c43-a3e3-d50f3e7dadbf None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:35:06.910796 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5d9a921c-ac2e-4c43-a3e3-d50f3e7dadbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:06.934572 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:35:07.010676 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1191/2419] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:35:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:35:07.024421 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e57e5-6cb1-4ec6-ac4b-c05275758fab None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:07.025202 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e57e5-6cb1-4ec6-ac4b-c05275758fab None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:07.025315 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e57e5-6cb1-4ec6-ac4b-c05275758fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:07.054347 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-359e57e5-6cb1-4ec6-ac4b-c05275758fab None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:35:07.055752 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-359e57e5-6cb1-4ec6-ac4b-c05275758fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:07.064823 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1229/2420] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:35:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:35:07.070620 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a921c-ac2e-4c43-a3e3-d50f3e7dadbf None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:07.082624 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a921c-ac2e-4c43-a3e3-d50f3e7dadbf None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:07.090618 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b7c092-bf24-4953-8d15-69552f00b956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:07.091439 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b7c092-bf24-4953-8d15-69552f00b956 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:07.091833 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b7c092-bf24-4953-8d15-69552f00b956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:07.092121 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b7c092-bf24-4953-8d15-69552f00b956 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:07.093236 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b7c092-bf24-4953-8d15-69552f00b956 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:35:07.093607 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b7c092-bf24-4953-8d15-69552f00b956 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:07.095697 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b7c092-bf24-4953-8d15-69552f00b956 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:07.110159 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1192/2421] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:35:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:35:07.115431 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e57e5-6cb1-4ec6-ac4b-c05275758fab None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:07.129771 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e57e5-6cb1-4ec6-ac4b-c05275758fab None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:07.136590 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55727b54-2099-4b42-b242-a3b0b58a142f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:07.137351 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55727b54-2099-4b42-b242-a3b0b58a142f admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:07.137734 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55727b54-2099-4b42-b242-a3b0b58a142f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:07.138144 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55727b54-2099-4b42-b242-a3b0b58a142f admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:07.138939 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55727b54-2099-4b42-b242-a3b0b58a142f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:07.141102 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55727b54-2099-4b42-b242-a3b0b58a142f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:07.199827 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1230/2422] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:35:07 2022] POST /identity/v3/projects => generated 385 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:35:07.205757 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b7c092-bf24-4953-8d15-69552f00b956 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:07.218436 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b7c092-bf24-4953-8d15-69552f00b956 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:07.300534 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1103d31a-00d0-41d9-8533-5cf2ae0ffa4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:07.301108 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103d31a-00d0-41d9-8533-5cf2ae0ffa4d admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:07.301502 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103d31a-00d0-41d9-8533-5cf2ae0ffa4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:07.301995 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103d31a-00d0-41d9-8533-5cf2ae0ffa4d admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:07.302996 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1103d31a-00d0-41d9-8533-5cf2ae0ffa4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:07.305148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1103d31a-00d0-41d9-8533-5cf2ae0ffa4d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:07.337409 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1193/2423] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:35:07 2022] POST /identity/v3/users => generated 293 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:35:07.347503 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55727b54-2099-4b42-b242-a3b0b58a142f admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:07.360308 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55727b54-2099-4b42-b242-a3b0b58a142f admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:07.366361 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200a3e7e-7b1f-46ca-aec4-a6afc2e56802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:07.367062 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a3e7e-7b1f-46ca-aec4-a6afc2e56802 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:07.367344 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a3e7e-7b1f-46ca-aec4-a6afc2e56802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:07.367617 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a3e7e-7b1f-46ca-aec4-a6afc2e56802 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:07.368397 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200a3e7e-7b1f-46ca-aec4-a6afc2e56802 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:07.370132 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200a3e7e-7b1f-46ca-aec4-a6afc2e56802 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:07.385514 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1231/2424] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:35:07.393375 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1103d31a-00d0-41d9-8533-5cf2ae0ffa4d admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:07.405519 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1103d31a-00d0-41d9-8533-5cf2ae0ffa4d admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:07.414691 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da923d1-4b11-47c2-8863-b0f4de9127bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:07.415907 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da923d1-4b11-47c2-8863-b0f4de9127bd admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:07.416441 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da923d1-4b11-47c2-8863-b0f4de9127bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:07.417038 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da923d1-4b11-47c2-8863-b0f4de9127bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:07.418911 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da923d1-4b11-47c2-8863-b0f4de9127bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:07.422286 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da923d1-4b11-47c2-8863-b0f4de9127bd admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:07.442106 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1194/2425] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:35:07.447071 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200a3e7e-7b1f-46ca-aec4-a6afc2e56802 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:07.465217 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200a3e7e-7b1f-46ca-aec4-a6afc2e56802 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:07.475081 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb92f5f-7d16-4652-8708-f73e26912b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:07.476762 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb92f5f-7d16-4652-8708-f73e26912b1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:07.477153 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb92f5f-7d16-4652-8708-f73e26912b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:07.477568 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb92f5f-7d16-4652-8708-f73e26912b1f admin admin] PATH_INFO: `/v3/projects/b466c8d09c724e26a0883bd5315593c9/users/7220499985644eb3804911f12c018b88/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:07.500777 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb92f5f-7d16-4652-8708-f73e26912b1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b466c8d09c724e26a0883bd5315593c9, user_id=7220499985644eb3804911f12c018b88, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:07.503657 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb92f5f-7d16-4652-8708-f73e26912b1f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:07.522978 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1232/2426] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:35:07 2022] PUT /identity/v3/projects/b466c8d09c724e26a0883bd5315593c9/users/7220499985644eb3804911f12c018b88/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:35:07.530697 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c42c6e7-00d2-4008-89a3-f481165e4a71 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:07.531101 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c42c6e7-00d2-4008-89a3-f481165e4a71 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:07.531637 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c42c6e7-00d2-4008-89a3-f481165e4a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:07.562847 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3c42c6e7-00d2-4008-89a3-f481165e4a71 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:35:07.564516 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3c42c6e7-00d2-4008-89a3-f481165e4a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:07.665770 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1195/2427] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:35:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:35:07.681914 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb92f5f-7d16-4652-8708-f73e26912b1f admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:07.686296 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-cfb92f5f-7d16-4652-8708-f73e26912b1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:07.709401 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:35:07.815891 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb92f5f-7d16-4652-8708-f73e26912b1f admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:07.823718 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f24f2d0-37b6-429f-b7c6-e2391811ae0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:07.824852 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24f2d0-37b6-429f-b7c6-e2391811ae0c service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:07.825329 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24f2d0-37b6-429f-b7c6-e2391811ae0c service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:07.825707 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24f2d0-37b6-429f-b7c6-e2391811ae0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:07.841052 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f24f2d0-37b6-429f-b7c6-e2391811ae0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:07.843961 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f24f2d0-37b6-429f-b7c6-e2391811ae0c service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:07.866885 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1233/2428] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:35:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:35:14.154005 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f91117-7bce-49db-b193-b602ffa7e7be None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:14.154835 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f91117-7bce-49db-b193-b602ffa7e7be None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:14.154998 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f91117-7bce-49db-b193-b602ffa7e7be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:14.193168 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-05f91117-7bce-49db-b193-b602ffa7e7be None None] MFA Rules not processed for user `7220499985644eb3804911f12c018b88`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:35:14.194885 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-05f91117-7bce-49db-b193-b602ffa7e7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:14.226591 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:35:14.328825 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1196/2429] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:35:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:35:14.330942 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f24f2d0-37b6-429f-b7c6-e2391811ae0c service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:14.353609 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f24f2d0-37b6-429f-b7c6-e2391811ae0c service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:14.362986 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c5045-4728-47c0-bc7d-e8a1e20b6891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:14.364319 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c5045-4728-47c0-bc7d-e8a1e20b6891 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:14.365174 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c5045-4728-47c0-bc7d-e8a1e20b6891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:14.365899 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c5045-4728-47c0-bc7d-e8a1e20b6891 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:14.367592 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197c5045-4728-47c0-bc7d-e8a1e20b6891 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:14.371734 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197c5045-4728-47c0-bc7d-e8a1e20b6891 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:14.420855 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1234/2430] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:35:14 2022] POST /identity/v3/projects => generated 381 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:35:14.428450 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f91117-7bce-49db-b193-b602ffa7e7be None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:14.449944 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f91117-7bce-49db-b193-b602ffa7e7be None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:14.602093 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e258e508-15d8-469f-8b76-e0460a6069f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:14.602093 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258e508-15d8-469f-8b76-e0460a6069f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:14.602556 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258e508-15d8-469f-8b76-e0460a6069f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:14.603693 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258e508-15d8-469f-8b76-e0460a6069f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:14.604668 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e258e508-15d8-469f-8b76-e0460a6069f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:14.608647 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e258e508-15d8-469f-8b76-e0460a6069f7 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:14.651987 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1197/2431] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:35:14 2022] POST /identity/v3/users => generated 297 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:35:14.657846 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c5045-4728-47c0-bc7d-e8a1e20b6891 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:14.718080 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c5045-4728-47c0-bc7d-e8a1e20b6891 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:14.729501 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ebed58-874a-4b1a-9976-9645064a7079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:14.731187 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ebed58-874a-4b1a-9976-9645064a7079 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:14.731857 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ebed58-874a-4b1a-9976-9645064a7079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:14.732324 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ebed58-874a-4b1a-9976-9645064a7079 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:14.733275 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ebed58-874a-4b1a-9976-9645064a7079 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:14.738407 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ebed58-874a-4b1a-9976-9645064a7079 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:14.766077 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1235/2432] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:35:14.772893 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e258e508-15d8-469f-8b76-e0460a6069f7 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:14.789736 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e258e508-15d8-469f-8b76-e0460a6069f7 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:14.835322 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda1c661-90b6-4b81-b9eb-2acbfb7bb414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:14.836848 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1c661-90b6-4b81-b9eb-2acbfb7bb414 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:14.837313 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1c661-90b6-4b81-b9eb-2acbfb7bb414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:14.841157 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1c661-90b6-4b81-b9eb-2acbfb7bb414 admin admin] PATH_INFO: `/v3/projects/2cf9cf3dd84b4107b431405d3c2a287b/users/f09cac1e3a4e4430a055cebab7a7ce9f/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:14.895413 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda1c661-90b6-4b81-b9eb-2acbfb7bb414 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cf9cf3dd84b4107b431405d3c2a287b, user_id=f09cac1e3a4e4430a055cebab7a7ce9f, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:14.901449 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda1c661-90b6-4b81-b9eb-2acbfb7bb414 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:14.930195 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1198/2433] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:35:14 2022] PUT /identity/v3/projects/2cf9cf3dd84b4107b431405d3c2a287b/users/f09cac1e3a4e4430a055cebab7a7ce9f/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:35:14.936525 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ebed58-874a-4b1a-9976-9645064a7079 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:14.964464 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ebed58-874a-4b1a-9976-9645064a7079 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:15.090761 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c503cea8-6f40-489e-abf6-1aea1c2aaea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:15.091867 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503cea8-6f40-489e-abf6-1aea1c2aaea6 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:15.092230 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503cea8-6f40-489e-abf6-1aea1c2aaea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:15.092633 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503cea8-6f40-489e-abf6-1aea1c2aaea6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:15.093427 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c503cea8-6f40-489e-abf6-1aea1c2aaea6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:15.095521 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c503cea8-6f40-489e-abf6-1aea1c2aaea6 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:15.108243 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1236/2434] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:35:15.112621 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda1c661-90b6-4b81-b9eb-2acbfb7bb414 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:15.131333 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda1c661-90b6-4b81-b9eb-2acbfb7bb414 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:15.137299 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d73e63-c104-49ef-b628-50b8cb301f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:15.138491 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d73e63-c104-49ef-b628-50b8cb301f2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:15.138818 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d73e63-c104-49ef-b628-50b8cb301f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:15.139056 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d73e63-c104-49ef-b628-50b8cb301f2f admin admin] PATH_INFO: `/v3/projects/2cf9cf3dd84b4107b431405d3c2a287b/users/f09cac1e3a4e4430a055cebab7a7ce9f/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:15.141431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d73e63-c104-49ef-b628-50b8cb301f2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cf9cf3dd84b4107b431405d3c2a287b, user_id=f09cac1e3a4e4430a055cebab7a7ce9f, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:15.144773 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d73e63-c104-49ef-b628-50b8cb301f2f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:15.160356 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1199/2435] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:35:15 2022] PUT /identity/v3/projects/2cf9cf3dd84b4107b431405d3c2a287b/users/f09cac1e3a4e4430a055cebab7a7ce9f/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:35:22.202832 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f68896-db0b-4422-b828-3343ef2e5301 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:22.204037 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f68896-db0b-4422-b828-3343ef2e5301 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:22.204037 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f68896-db0b-4422-b828-3343ef2e5301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:22.248128 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-21f68896-db0b-4422-b828-3343ef2e5301 None None] MFA Rules not processed for user `f09cac1e3a4e4430a055cebab7a7ce9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:35:22.249594 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-21f68896-db0b-4422-b828-3343ef2e5301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:22.360768 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1237/2436] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:35:22 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:35:22.377605 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d73e63-c104-49ef-b628-50b8cb301f2f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:22.380655 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-98d73e63-c104-49ef-b628-50b8cb301f2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:22.471688 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d73e63-c104-49ef-b628-50b8cb301f2f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:22.478903 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af755db2-0a98-48ad-9226-5f291a5a3921 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7da4bf6f3be47b4aa25c6240cb4be03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:22.480159 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af755db2-0a98-48ad-9226-5f291a5a3921 service glance] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:22.480753 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af755db2-0a98-48ad-9226-5f291a5a3921 service glance] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:22.481337 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af755db2-0a98-48ad-9226-5f291a5a3921 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:22.520209 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:35:22.588954 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af755db2-0a98-48ad-9226-5f291a5a3921 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:22.591034 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af755db2-0a98-48ad-9226-5f291a5a3921 service glance] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:22.603281 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1200/2437] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:35:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:35:22.780182 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6556aa-c6c3-4b10-8c2e-5d9a5bb34a58 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:22.780589 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6556aa-c6c3-4b10-8c2e-5d9a5bb34a58 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:22.781060 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6556aa-c6c3-4b10-8c2e-5d9a5bb34a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:22.788763 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:35:22.806935 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-1b6556aa-c6c3-4b10-8c2e-5d9a5bb34a58 None None] MFA Rules not processed for user `09c9fa7dccb8491c9ad3b43967345ea2`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:35:22.808401 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-1b6556aa-c6c3-4b10-8c2e-5d9a5bb34a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:22.909019 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1238/2438] 10.222.0.87 () {58 vars in 1009 bytes} [Tue Jun 21 18:35:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:35:23.001691 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af755db2-0a98-48ad-9226-5f291a5a3921 service glance] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:23.003899 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-af755db2-0a98-48ad-9226-5f291a5a3921 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:23.120735 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af755db2-0a98-48ad-9226-5f291a5a3921 service glance] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:23.127777 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cd8c95-2863-425c-b911-a511b6f7128e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '515b3571045e4fe29565307c8ecc10b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:23.128605 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cd8c95-2863-425c-b911-a511b6f7128e service swift] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:23.128961 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cd8c95-2863-425c-b911-a511b6f7128e service swift] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:23.129308 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cd8c95-2863-425c-b911-a511b6f7128e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:23.140771 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cd8c95-2863-425c-b911-a511b6f7128e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:23.143333 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cd8c95-2863-425c-b911-a511b6f7128e service swift] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:23.156872 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1201/2439] 10.222.0.87 () {60 vars in 1384 bytes} [Tue Jun 21 18:35:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 18:35:32.901193 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=24 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:35:33.150529 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=48 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:35:34.837959 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2f728-e5b8-4624-8f46-86e4b70efe0f None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:34.838858 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2f728-e5b8-4624-8f46-86e4b70efe0f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:34.838858 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2f728-e5b8-4624-8f46-86e4b70efe0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:34.862811 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-11a2f728-e5b8-4624-8f46-86e4b70efe0f None None] MFA Rules not processed for user `7220499985644eb3804911f12c018b88`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:35:34.864679 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-11a2f728-e5b8-4624-8f46-86e4b70efe0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:34.872990 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1239/2440] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:35:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:35:37.571149 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd29d8-54f1-41b1-8cb4-66e4dc9bb307 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:37.572022 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd29d8-54f1-41b1-8cb4-66e4dc9bb307 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:37.572185 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd29d8-54f1-41b1-8cb4-66e4dc9bb307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:37.609188 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-64dd29d8-54f1-41b1-8cb4-66e4dc9bb307 None None] MFA Rules not processed for user `09c9fa7dccb8491c9ad3b43967345ea2`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:35:37.611676 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-64dd29d8-54f1-41b1-8cb4-66e4dc9bb307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:37.625913 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1202/2441] 10.222.0.87 () {58 vars in 1009 bytes} [Tue Jun 21 18:35:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:35:37.646366 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a2f728-e5b8-4624-8f46-86e4b70efe0f None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:37.663213 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a2f728-e5b8-4624-8f46-86e4b70efe0f None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:37.670778 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce5533-f9b2-4bc2-b639-b8ed151931a0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '515b3571045e4fe29565307c8ecc10b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:37.671718 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce5533-f9b2-4bc2-b639-b8ed151931a0 service swift] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:37.672096 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce5533-f9b2-4bc2-b639-b8ed151931a0 service swift] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:37.672370 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce5533-f9b2-4bc2-b639-b8ed151931a0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:37.684414 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ce5533-f9b2-4bc2-b639-b8ed151931a0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:37.686774 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ce5533-f9b2-4bc2-b639-b8ed151931a0 service swift] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:37.700982 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1240/2442] 10.222.0.87 () {60 vars in 1384 bytes} [Tue Jun 21 18:35:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 21 18:35:46.444428 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dd29d8-54f1-41b1-8cb4-66e4dc9bb307 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:46.449690 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-64dd29d8-54f1-41b1-8cb4-66e4dc9bb307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:46.599293 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dd29d8-54f1-41b1-8cb4-66e4dc9bb307 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:46.610153 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c1b260-dba3-4bef-81e5-5407a72362db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:46.611847 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c1b260-dba3-4bef-81e5-5407a72362db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:46.612342 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c1b260-dba3-4bef-81e5-5407a72362db admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:46.612840 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c1b260-dba3-4bef-81e5-5407a72362db admin admin] PATH_INFO: `/v3/users/ec179985fc3b41d188ca69d9e03263d0` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:46.615445 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c1b260-dba3-4bef-81e5-5407a72362db admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec179985fc3b41d188ca69d9e03263d0)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:46.619725 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c1b260-dba3-4bef-81e5-5407a72362db admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:46.745479 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a5c1b260-dba3-4bef-81e5-5407a72362db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec179985fc3b41d188ca69d9e03263d0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:46.769482 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a5c1b260-dba3-4bef-81e5-5407a72362db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec179985fc3b41d188ca69d9e03263d0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:46.771255 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a5c1b260-dba3-4bef-81e5-5407a72362db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec179985fc3b41d188ca69d9e03263d0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:46.774752 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a5c1b260-dba3-4bef-81e5-5407a72362db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec179985fc3b41d188ca69d9e03263d0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:46.803235 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a5c1b260-dba3-4bef-81e5-5407a72362db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec179985fc3b41d188ca69d9e03263d0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:46.838830 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1203/2443] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:35:46 2022] DELETE /identity/v3/users/ec179985fc3b41d188ca69d9e03263d0 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:35:47.237188 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce5533-f9b2-4bc2-b639-b8ed151931a0 service swift] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:47.258476 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce5533-f9b2-4bc2-b639-b8ed151931a0 service swift] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:47.464815 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ed5af-7eec-45c1-8243-81bf40c1b5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:47.466366 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ed5af-7eec-45c1-8243-81bf40c1b5c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:47.467115 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ed5af-7eec-45c1-8243-81bf40c1b5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:47.468086 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ed5af-7eec-45c1-8243-81bf40c1b5c0 admin admin] PATH_INFO: `/v3/projects/55c4efeea2f54994b6f45b5cc346c43b` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:47.470769 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121ed5af-7eec-45c1-8243-81bf40c1b5c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55c4efeea2f54994b6f45b5cc346c43b)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:47.474557 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121ed5af-7eec-45c1-8243-81bf40c1b5c0 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:47.592111 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-121ed5af-7eec-45c1-8243-81bf40c1b5c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55c4efeea2f54994b6f45b5cc346c43b', 'request_id': 'req-121ed5af-7eec-45c1-8243-81bf40c1b5c0'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:47.604531 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-121ed5af-7eec-45c1-8243-81bf40c1b5c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55c4efeea2f54994b6f45b5cc346c43b', 'request_id': 'req-121ed5af-7eec-45c1-8243-81bf40c1b5c0'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:47.623333 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-121ed5af-7eec-45c1-8243-81bf40c1b5c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55c4efeea2f54994b6f45b5cc346c43b', 'request_id': 'req-121ed5af-7eec-45c1-8243-81bf40c1b5c0'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:47.647991 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-121ed5af-7eec-45c1-8243-81bf40c1b5c0 admin admin] The token cache is being invalidate because project 55c4efeea2f54994b6f45b5cc346c43b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:35:47.648502 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-121ed5af-7eec-45c1-8243-81bf40c1b5c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:47.649796 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-121ed5af-7eec-45c1-8243-81bf40c1b5c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:47.652888 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1241/2444] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:35:47 2022] DELETE /identity/v3/projects/55c4efeea2f54994b6f45b5cc346c43b => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:35:49.290724 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c1b260-dba3-4bef-81e5-5407a72362db admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:49.293170 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a5c1b260-dba3-4bef-81e5-5407a72362db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:49.419106 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c1b260-dba3-4bef-81e5-5407a72362db admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:49.437265 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:35:49.531721 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3820d9ac-94e2-43ec-ba8f-02f11b647b6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:49.532690 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3820d9ac-94e2-43ec-ba8f-02f11b647b6c service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:49.532979 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3820d9ac-94e2-43ec-ba8f-02f11b647b6c service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:49.533256 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3820d9ac-94e2-43ec-ba8f-02f11b647b6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:49.535375 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3820d9ac-94e2-43ec-ba8f-02f11b647b6c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:49.644852 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3820d9ac-94e2-43ec-ba8f-02f11b647b6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:49.648116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3820d9ac-94e2-43ec-ba8f-02f11b647b6c service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:49.667736 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1204/2445] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:35:49 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:35:52.883974 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ed5af-7eec-45c1-8243-81bf40c1b5c0 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:52.886214 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-121ed5af-7eec-45c1-8243-81bf40c1b5c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:52.918650 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ed5af-7eec-45c1-8243-81bf40c1b5c0 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:52.939700 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e16e83f-3261-4920-beb0-0bde98f35bcc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:52.940825 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e16e83f-3261-4920-beb0-0bde98f35bcc service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:52.941289 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e16e83f-3261-4920-beb0-0bde98f35bcc service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:52.942031 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e16e83f-3261-4920-beb0-0bde98f35bcc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:52.947367 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8e16e83f-3261-4920-beb0-0bde98f35bcc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:52.980196 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e16e83f-3261-4920-beb0-0bde98f35bcc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:52.983759 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e16e83f-3261-4920-beb0-0bde98f35bcc service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:53.004939 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1242/2446] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:35:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:35:59.659050 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=51 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:36:00.588152 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3820d9ac-94e2-43ec-ba8f-02f11b647b6c service nova] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:36:00.600601 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3820d9ac-94e2-43ec-ba8f-02f11b647b6c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:36:00.629672 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3820d9ac-94e2-43ec-ba8f-02f11b647b6c service nova] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:36:00.641223 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e192c7-a813-41f7-b4db-bd119be9b72a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:36:00.642727 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e192c7-a813-41f7-b4db-bd119be9b72a service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:00.643363 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e192c7-a813-41f7-b4db-bd119be9b72a service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:00.643959 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e192c7-a813-41f7-b4db-bd119be9b72a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:00.647600 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d2e192c7-a813-41f7-b4db-bd119be9b72a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:36:00.670827 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e192c7-a813-41f7-b4db-bd119be9b72a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:36:00.673260 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e192c7-a813-41f7-b4db-bd119be9b72a service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:36:00.686587 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1205/2447] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:36:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:36:02.995843 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=66,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:36:10.680961 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=6 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:36:15.129339 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e16e83f-3261-4920-beb0-0bde98f35bcc service nova] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:36:15.131461 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8e16e83f-3261-4920-beb0-0bde98f35bcc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:36:15.236213 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e16e83f-3261-4920-beb0-0bde98f35bcc service nova] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:36:15.242660 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ce9551-df5a-4a65-af4e-25397237bb5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:36:15.243778 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce9551-df5a-4a65-af4e-25397237bb5d service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:15.244283 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce9551-df5a-4a65-af4e-25397237bb5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:15.244787 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce9551-df5a-4a65-af4e-25397237bb5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:15.248164 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-88ce9551-df5a-4a65-af4e-25397237bb5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:36:15.381753 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ce9551-df5a-4a65-af4e-25397237bb5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:36:15.383447 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ce9551-df5a-4a65-af4e-25397237bb5d service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:36:15.398248 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1243/2448] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:36:15 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:36:18.133046 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e192c7-a813-41f7-b4db-bd119be9b72a service nova] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:36:18.150807 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e192c7-a813-41f7-b4db-bd119be9b72a service nova] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:36:18.160451 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605c5166-c371-45d4-a0ed-66d450581205 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:36:18.161934 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c5166-c371-45d4-a0ed-66d450581205 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:18.162385 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c5166-c371-45d4-a0ed-66d450581205 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:18.162815 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c5166-c371-45d4-a0ed-66d450581205 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:18.165866 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-605c5166-c371-45d4-a0ed-66d450581205 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:36:18.188876 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605c5166-c371-45d4-a0ed-66d450581205 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:36:18.192063 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605c5166-c371-45d4-a0ed-66d450581205 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:36:18.212142 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1206/2449] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:36:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:36:22.783501 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637c88a-b677-4a0a-91c6-da6325b11eaf None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:22.784935 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637c88a-b677-4a0a-91c6-da6325b11eaf None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:22.786437 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637c88a-b677-4a0a-91c6-da6325b11eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:22.821074 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-c637c88a-b677-4a0a-91c6-da6325b11eaf None None] MFA Rules not processed for user `de09b91800314d06aeddafe3c12a77b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:36:22.823666 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c637c88a-b677-4a0a-91c6-da6325b11eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:36:22.834914 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1244/2450] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:36:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:36:28.204974 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=6 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:36:32.817534 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=53 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:37:00.853818 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c66328-3835-49b6-8407-3a4b41fb8e10 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:37:00.854596 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c66328-3835-49b6-8407-3a4b41fb8e10 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:37:00.854596 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c66328-3835-49b6-8407-3a4b41fb8e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:37:00.877226 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-79c66328-3835-49b6-8407-3a4b41fb8e10 None None] MFA Rules not processed for user `144c9c31ed9b469aa200dcb16c89ab4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:37:00.878523 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-79c66328-3835-49b6-8407-3a4b41fb8e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:37:00.955838 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1207/2451] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:37:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:37:10.943043 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=25 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:37:43.504085 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c637c88a-b677-4a0a-91c6-da6325b11eaf None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:37:43.507167 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c637c88a-b677-4a0a-91c6-da6325b11eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:37:43.627661 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c637c88a-b677-4a0a-91c6-da6325b11eaf None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:37:43.633556 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa4ce41-0e8b-40d1-a3ac-51f87c3203be service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef1d49b36cdc4665b82487be97e9dcd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:37:43.634446 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa4ce41-0e8b-40d1-a3ac-51f87c3203be service placement] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:37:43.634954 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa4ce41-0e8b-40d1-a3ac-51f87c3203be service placement] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:37:43.636225 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa4ce41-0e8b-40d1-a3ac-51f87c3203be service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:37:43.638294 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3aa4ce41-0e8b-40d1-a3ac-51f87c3203be service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:37:43.652059 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa4ce41-0e8b-40d1-a3ac-51f87c3203be service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:37:43.654200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa4ce41-0e8b-40d1-a3ac-51f87c3203be service placement] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:37:43.669896 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1245/2452] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:37:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:37:53.502655 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c66328-3835-49b6-8407-3a4b41fb8e10 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:37:53.517021 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c66328-3835-49b6-8407-3a4b41fb8e10 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:37:53.523712 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbafe89c-89aa-4206-b319-a688139c8ca4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:37:53.525314 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbafe89c-89aa-4206-b319-a688139c8ca4 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:37:53.525676 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbafe89c-89aa-4206-b319-a688139c8ca4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:37:53.525874 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbafe89c-89aa-4206-b319-a688139c8ca4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:37:53.541930 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fbafe89c-89aa-4206-b319-a688139c8ca4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:37:53.661772 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=32 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:37:53.666050 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbafe89c-89aa-4206-b319-a688139c8ca4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:37:53.667800 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbafe89c-89aa-4206-b319-a688139c8ca4 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:37:53.680351 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1208/2453] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:37:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:38:00.190603 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa4ce41-0e8b-40d1-a3ac-51f87c3203be service placement] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:38:00.194284 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3aa4ce41-0e8b-40d1-a3ac-51f87c3203be service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:38:00.231339 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa4ce41-0e8b-40d1-a3ac-51f87c3203be service placement] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:38:00.244710 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b23d90f-bf14-4b1e-927b-7c8db17299fe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef1d49b36cdc4665b82487be97e9dcd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:38:00.247611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23d90f-bf14-4b1e-927b-7c8db17299fe service placement] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:38:00.249002 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23d90f-bf14-4b1e-927b-7c8db17299fe service placement] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:38:00.249735 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23d90f-bf14-4b1e-927b-7c8db17299fe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:38:00.254043 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3b23d90f-bf14-4b1e-927b-7c8db17299fe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:38:00.280615 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b23d90f-bf14-4b1e-927b-7c8db17299fe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:38:00.283025 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b23d90f-bf14-4b1e-927b-7c8db17299fe service placement] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:38:00.325122 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1246/2454] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:38:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:38:00.433763 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbafe89c-89aa-4206-b319-a688139c8ca4 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:38:00.436121 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fbafe89c-89aa-4206-b319-a688139c8ca4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:38:00.491736 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbafe89c-89aa-4206-b319-a688139c8ca4 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:38:00.509512 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d656f7f-a25c-4049-a1e9-2a010c7774ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:38:00.511287 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d656f7f-a25c-4049-a1e9-2a010c7774ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:38:00.511915 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d656f7f-a25c-4049-a1e9-2a010c7774ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:38:00.512353 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d656f7f-a25c-4049-a1e9-2a010c7774ff admin admin] PATH_INFO: `/v3/users/de09b91800314d06aeddafe3c12a77b3` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:38:00.515554 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d656f7f-a25c-4049-a1e9-2a010c7774ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=de09b91800314d06aeddafe3c12a77b3)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:38:00.520800 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d656f7f-a25c-4049-a1e9-2a010c7774ff admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:38:00.613401 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-1d656f7f-a25c-4049-a1e9-2a010c7774ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de09b91800314d06aeddafe3c12a77b3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:38:00.628774 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-1d656f7f-a25c-4049-a1e9-2a010c7774ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de09b91800314d06aeddafe3c12a77b3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:38:00.630040 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-1d656f7f-a25c-4049-a1e9-2a010c7774ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de09b91800314d06aeddafe3c12a77b3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:38:00.631089 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-1d656f7f-a25c-4049-a1e9-2a010c7774ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de09b91800314d06aeddafe3c12a77b3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:38:00.651756 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-1d656f7f-a25c-4049-a1e9-2a010c7774ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de09b91800314d06aeddafe3c12a77b3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:38:00.674119 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1209/2455] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:38:00 2022] DELETE /identity/v3/users/de09b91800314d06aeddafe3c12a77b3 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:38:00.956832 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b23d90f-bf14-4b1e-927b-7c8db17299fe service placement] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:38:00.980515 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b23d90f-bf14-4b1e-927b-7c8db17299fe service placement] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:38:01.074561 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43742648-2495-4381-8342-e7f1c1272b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:38:01.075430 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43742648-2495-4381-8342-e7f1c1272b73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:38:01.075718 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43742648-2495-4381-8342-e7f1c1272b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:38:01.075998 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43742648-2495-4381-8342-e7f1c1272b73 admin admin] PATH_INFO: `/v3/projects/6538654ef94d4036a7a8d5614b80b0ed` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:38:01.077394 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43742648-2495-4381-8342-e7f1c1272b73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6538654ef94d4036a7a8d5614b80b0ed)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:38:01.079503 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43742648-2495-4381-8342-e7f1c1272b73 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:38:01.151195 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-43742648-2495-4381-8342-e7f1c1272b73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6538654ef94d4036a7a8d5614b80b0ed', 'request_id': 'req-43742648-2495-4381-8342-e7f1c1272b73'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:38:01.160487 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-43742648-2495-4381-8342-e7f1c1272b73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6538654ef94d4036a7a8d5614b80b0ed', 'request_id': 'req-43742648-2495-4381-8342-e7f1c1272b73'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:38:01.173682 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-43742648-2495-4381-8342-e7f1c1272b73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6538654ef94d4036a7a8d5614b80b0ed', 'request_id': 'req-43742648-2495-4381-8342-e7f1c1272b73'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:38:01.196167 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-43742648-2495-4381-8342-e7f1c1272b73 admin admin] The token cache is being invalidate because project 6538654ef94d4036a7a8d5614b80b0ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:38:01.197777 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-43742648-2495-4381-8342-e7f1c1272b73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:38:01.198737 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-43742648-2495-4381-8342-e7f1c1272b73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:38:01.201898 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1247/2456] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:38:00 2022] DELETE /identity/v3/projects/6538654ef94d4036a7a8d5614b80b0ed => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:38:01.214035 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59579dd6-c597-4cf8-bd6a-c19bf4376d34 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:38:01.214532 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59579dd6-c597-4cf8-bd6a-c19bf4376d34 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:38:01.214973 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59579dd6-c597-4cf8-bd6a-c19bf4376d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:38:01.245954 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-59579dd6-c597-4cf8-bd6a-c19bf4376d34 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:38:01.248739 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-59579dd6-c597-4cf8-bd6a-c19bf4376d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:38:01.371116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1210/2457] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:38:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:38:01.381385 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660db1b-3005-4d7a-814c-2966a9f06f13 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:38:01.381977 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660db1b-3005-4d7a-814c-2966a9f06f13 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:38:01.382803 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660db1b-3005-4d7a-814c-2966a9f06f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:38:01.415578 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-a660db1b-3005-4d7a-814c-2966a9f06f13 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:38:01.417769 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a660db1b-3005-4d7a-814c-2966a9f06f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:38:01.432439 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1248/2458] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:38:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:38:01.438542 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59579dd6-c597-4cf8-bd6a-c19bf4376d34 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:38:01.450435 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59579dd6-c597-4cf8-bd6a-c19bf4376d34 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:38:01.455682 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dfd867-1ae9-4348-af5f-845d2cb17055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:38:01.456270 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfd867-1ae9-4348-af5f-845d2cb17055 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:38:01.456948 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfd867-1ae9-4348-af5f-845d2cb17055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:38:01.456948 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfd867-1ae9-4348-af5f-845d2cb17055 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:38:01.457589 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dfd867-1ae9-4348-af5f-845d2cb17055 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:38:01.457939 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dfd867-1ae9-4348-af5f-845d2cb17055 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:38:01.460089 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dfd867-1ae9-4348-af5f-845d2cb17055 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:38:01.463644 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:38:01.490333 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1211/2459] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:38:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:38:01.491222 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a660db1b-3005-4d7a-814c-2966a9f06f13 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:38:01.508774 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a660db1b-3005-4d7a-814c-2966a9f06f13 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:38:01.519033 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0714b8-14c1-4166-bc20-0a7b2129178b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:38:01.520197 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0714b8-14c1-4166-bc20-0a7b2129178b admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:38:01.520709 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0714b8-14c1-4166-bc20-0a7b2129178b admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:38:01.521281 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0714b8-14c1-4166-bc20-0a7b2129178b admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:38:01.522821 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0714b8-14c1-4166-bc20-0a7b2129178b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:38:01.526318 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0714b8-14c1-4166-bc20-0a7b2129178b admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:38:01.586585 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1249/2460] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:38:01 2022] POST /identity/v3/projects => generated 389 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:38:01.592223 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dfd867-1ae9-4348-af5f-845d2cb17055 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:38:01.611070 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dfd867-1ae9-4348-af5f-845d2cb17055 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:38:01.728036 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda127e-aff1-4771-b074-116f097567e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:38:01.729620 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda127e-aff1-4771-b074-116f097567e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:38:01.730392 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda127e-aff1-4771-b074-116f097567e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:38:01.731150 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda127e-aff1-4771-b074-116f097567e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:38:01.733028 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edda127e-aff1-4771-b074-116f097567e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:38:01.737710 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edda127e-aff1-4771-b074-116f097567e0 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:38:01.779320 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1212/2461] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:38:01 2022] POST /identity/v3/users => generated 295 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:38:01.784448 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0714b8-14c1-4166-bc20-0a7b2129178b admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:38:01.802198 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0714b8-14c1-4166-bc20-0a7b2129178b admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:38:01.825477 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e2bde-17fa-4fc3-87b1-492b8e29c79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:38:01.826744 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e2bde-17fa-4fc3-87b1-492b8e29c79e admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:38:01.827305 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e2bde-17fa-4fc3-87b1-492b8e29c79e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:38:01.828018 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e2bde-17fa-4fc3-87b1-492b8e29c79e admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:38:01.829239 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5e2bde-17fa-4fc3-87b1-492b8e29c79e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:38:01.831973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5e2bde-17fa-4fc3-87b1-492b8e29c79e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:38:01.861604 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1250/2462] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:38:01.864730 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda127e-aff1-4771-b074-116f097567e0 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:38:01.916015 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda127e-aff1-4771-b074-116f097567e0 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:38:01.925264 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cf378b-05b2-4398-806d-a13e4c10abc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:38:01.926703 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf378b-05b2-4398-806d-a13e4c10abc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:38:01.927280 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf378b-05b2-4398-806d-a13e4c10abc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:38:01.927759 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf378b-05b2-4398-806d-a13e4c10abc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:38:01.929897 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cf378b-05b2-4398-806d-a13e4c10abc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:38:01.933398 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cf378b-05b2-4398-806d-a13e4c10abc5 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:38:01.963716 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1213/2463] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:38:01.970636 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e2bde-17fa-4fc3-87b1-492b8e29c79e admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:38:01.988309 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e2bde-17fa-4fc3-87b1-492b8e29c79e admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:38:01.997234 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca803f03-5e0c-41a1-b97a-9fe2a4be1e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:38:01.998955 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca803f03-5e0c-41a1-b97a-9fe2a4be1e13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:38:01.999468 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca803f03-5e0c-41a1-b97a-9fe2a4be1e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:38:01.999963 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca803f03-5e0c-41a1-b97a-9fe2a4be1e13 admin admin] PATH_INFO: `/v3/projects/21a4c114c12d40abb896e2637ee6ab83/users/93785710c10749f9aa3d29b43965909c/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:38:02.030836 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca803f03-5e0c-41a1-b97a-9fe2a4be1e13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21a4c114c12d40abb896e2637ee6ab83, user_id=93785710c10749f9aa3d29b43965909c, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:38:02.036134 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca803f03-5e0c-41a1-b97a-9fe2a4be1e13 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:38:02.067980 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1251/2464] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:38:01 2022] PUT /identity/v3/projects/21a4c114c12d40abb896e2637ee6ab83/users/93785710c10749f9aa3d29b43965909c/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:38:02.076535 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d45628-8956-429f-9c53-a59df7ed4054 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:38:02.077057 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d45628-8956-429f-9c53-a59df7ed4054 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:38:02.077584 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d45628-8956-429f-9c53-a59df7ed4054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:38:02.105202 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-62d45628-8956-429f-9c53-a59df7ed4054 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:38:02.107661 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-62d45628-8956-429f-9c53-a59df7ed4054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:38:02.231109 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1214/2465] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:38:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:38:02.243905 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca803f03-5e0c-41a1-b97a-9fe2a4be1e13 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:38:02.247347 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ca803f03-5e0c-41a1-b97a-9fe2a4be1e13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:38:02.273621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:38:02.360887 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca803f03-5e0c-41a1-b97a-9fe2a4be1e13 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:38:02.365461 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba45646-3519-4fa2-8503-c80460e3b6b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:38:02.366162 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba45646-3519-4fa2-8503-c80460e3b6b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:38:02.366512 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba45646-3519-4fa2-8503-c80460e3b6b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:38:02.366859 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba45646-3519-4fa2-8503-c80460e3b6b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:38:02.388855 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba45646-3519-4fa2-8503-c80460e3b6b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:38:02.391589 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba45646-3519-4fa2-8503-c80460e3b6b4 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:38:02.406070 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1252/2466] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:38:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:38:05.018134 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b004d27f-2b99-4be6-aab1-a4898047b21c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:38:05.018940 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b004d27f-2b99-4be6-aab1-a4898047b21c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:38:05.019174 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b004d27f-2b99-4be6-aab1-a4898047b21c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:38:05.044284 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b004d27f-2b99-4be6-aab1-a4898047b21c None None] MFA Rules not processed for user `acf9a33e9fb941939010989a89a51d38`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:38:05.046306 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b004d27f-2b99-4be6-aab1-a4898047b21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:38:05.151850 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1215/2467] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:38:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:38:05.659610 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba45646-3519-4fa2-8503-c80460e3b6b4 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:38:05.675559 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba45646-3519-4fa2-8503-c80460e3b6b4 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:38:05.685960 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608859c5-08cd-419f-86e1-d1f0caf7171b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:38:05.687189 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608859c5-08cd-419f-86e1-d1f0caf7171b service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:38:05.687673 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608859c5-08cd-419f-86e1-d1f0caf7171b service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:38:05.688184 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608859c5-08cd-419f-86e1-d1f0caf7171b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:38:05.694681 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-608859c5-08cd-419f-86e1-d1f0caf7171b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:38:05.707073 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608859c5-08cd-419f-86e1-d1f0caf7171b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:38:05.708751 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608859c5-08cd-419f-86e1-d1f0caf7171b service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:38:05.726446 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1253/2468] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:38:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:38:09.918892 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a476d-6615-4f91-bbdb-c79d89a46d05 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:38:09.919718 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a476d-6615-4f91-bbdb-c79d89a46d05 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:38:09.919894 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a476d-6615-4f91-bbdb-c79d89a46d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:38:09.984444 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-2a0a476d-6615-4f91-bbdb-c79d89a46d05 None None] MFA Rules not processed for user `93785710c10749f9aa3d29b43965909c`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:38:09.987423 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2a0a476d-6615-4f91-bbdb-c79d89a46d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:38:10.001860 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:38:10.202238 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1216/2469] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:38:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:38:10.226246 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608859c5-08cd-419f-86e1-d1f0caf7171b service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:38:10.229972 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-608859c5-08cd-419f-86e1-d1f0caf7171b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:38:10.455116 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608859c5-08cd-419f-86e1-d1f0caf7171b service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:38:10.470475 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53600e58-fdcf-4c8b-91a0-d40392d94dd3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:38:10.472043 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53600e58-fdcf-4c8b-91a0-d40392d94dd3 service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:38:10.473010 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53600e58-fdcf-4c8b-91a0-d40392d94dd3 service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:38:10.473772 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53600e58-fdcf-4c8b-91a0-d40392d94dd3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:38:10.494061 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53600e58-fdcf-4c8b-91a0-d40392d94dd3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:38:10.497283 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53600e58-fdcf-4c8b-91a0-d40392d94dd3 service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:38:10.531759 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1254/2470] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:38:10 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:38:13.279318 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1458e-867b-45ec-82f9-49799a4fe00a None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:38:13.279318 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1458e-867b-45ec-82f9-49799a4fe00a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:38:13.279318 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1458e-867b-45ec-82f9-49799a4fe00a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:38:13.325971 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-5bc1458e-867b-45ec-82f9-49799a4fe00a None None] MFA Rules not processed for user `93785710c10749f9aa3d29b43965909c`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:38:13.329263 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5bc1458e-867b-45ec-82f9-49799a4fe00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:38:13.349167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1217/2471] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:38:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:38:20.513794 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=60 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:38:23.300904 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=27 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:38:59.172223 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b13a5-126e-40d1-9d75-b45982c65365 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:38:59.173831 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b13a5-126e-40d1-9d75-b45982c65365 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:38:59.174341 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b13a5-126e-40d1-9d75-b45982c65365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:38:59.203325 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-011b13a5-126e-40d1-9d75-b45982c65365 None None] MFA Rules not processed for user `144c9c31ed9b469aa200dcb16c89ab4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:38:59.205567 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-011b13a5-126e-40d1-9d75-b45982c65365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:38:59.306571 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1255/2472] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:38:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:39:04.534187 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0d893-e823-4b6f-a6a5-d24626bf5257 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:39:04.535131 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0d893-e823-4b6f-a6a5-d24626bf5257 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:39:04.535131 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0d893-e823-4b6f-a6a5-d24626bf5257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:39:04.559801 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-55b0d893-e823-4b6f-a6a5-d24626bf5257 None None] MFA Rules not processed for user `09c9fa7dccb8491c9ad3b43967345ea2`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:39:04.561370 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-55b0d893-e823-4b6f-a6a5-d24626bf5257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:39:04.661269 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1218/2473] 10.222.0.87 () {58 vars in 967 bytes} [Tue Jun 21 18:39:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:39:04.713924 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011b13a5-126e-40d1-9d75-b45982c65365 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:39:04.715410 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-011b13a5-126e-40d1-9d75-b45982c65365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:39:04.798914 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011b13a5-126e-40d1-9d75-b45982c65365 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:39:04.804457 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a766f85a-c8bd-4ed4-b806-7f1fac43e03c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '515b3571045e4fe29565307c8ecc10b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:39:04.805134 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766f85a-c8bd-4ed4-b806-7f1fac43e03c service swift] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:39:04.805422 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766f85a-c8bd-4ed4-b806-7f1fac43e03c service swift] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:39:04.805736 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766f85a-c8bd-4ed4-b806-7f1fac43e03c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:39:04.815862 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a766f85a-c8bd-4ed4-b806-7f1fac43e03c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:39:04.817515 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a766f85a-c8bd-4ed4-b806-7f1fac43e03c service swift] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:39:04.827778 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1256/2474] 10.222.0.87 () {60 vars in 1384 bytes} [Tue Jun 21 18:39:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 21 18:39:09.986861 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b0d893-e823-4b6f-a6a5-d24626bf5257 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:39:09.988834 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-55b0d893-e823-4b6f-a6a5-d24626bf5257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:39:10.119655 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b0d893-e823-4b6f-a6a5-d24626bf5257 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:39:10.130251 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc5b15-b6c4-41cd-98f1-396f211ffd89 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef1d49b36cdc4665b82487be97e9dcd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:39:10.131406 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc5b15-b6c4-41cd-98f1-396f211ffd89 service placement] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:39:10.131894 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc5b15-b6c4-41cd-98f1-396f211ffd89 service placement] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:39:10.133387 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc5b15-b6c4-41cd-98f1-396f211ffd89 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:39:10.137056 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8bcc5b15-b6c4-41cd-98f1-396f211ffd89 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:39:10.160420 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcc5b15-b6c4-41cd-98f1-396f211ffd89 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:39:10.164833 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcc5b15-b6c4-41cd-98f1-396f211ffd89 service placement] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:39:10.182108 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1219/2475] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:39:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:39:13.279545 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a766f85a-c8bd-4ed4-b806-7f1fac43e03c service swift] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:39:13.306552 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a766f85a-c8bd-4ed4-b806-7f1fac43e03c service swift] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:39:13.313190 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae0066d-bdd1-457e-b5ca-2597b936f890 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:39:13.314349 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0066d-bdd1-457e-b5ca-2597b936f890 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:39:13.314862 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0066d-bdd1-457e-b5ca-2597b936f890 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:39:13.315335 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0066d-bdd1-457e-b5ca-2597b936f890 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:39:13.319271 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3ae0066d-bdd1-457e-b5ca-2597b936f890 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:39:13.346920 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae0066d-bdd1-457e-b5ca-2597b936f890 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:39:13.350455 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae0066d-bdd1-457e-b5ca-2597b936f890 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:39:13.375503 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1257/2476] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:39:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:39:15.130432 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc5b15-b6c4-41cd-98f1-396f211ffd89 service placement] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:39:15.134079 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8bcc5b15-b6c4-41cd-98f1-396f211ffd89 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:39:15.166479 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc5b15-b6c4-41cd-98f1-396f211ffd89 service placement] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:39:15.177974 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e619ae14-5d7b-4950-be32-33dcd1d49865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:39:15.180006 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619ae14-5d7b-4950-be32-33dcd1d49865 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:39:15.180839 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619ae14-5d7b-4950-be32-33dcd1d49865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:39:15.181779 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619ae14-5d7b-4950-be32-33dcd1d49865 admin admin] PATH_INFO: `/v3/users/7220499985644eb3804911f12c018b88` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:39:15.185476 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e619ae14-5d7b-4950-be32-33dcd1d49865 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7220499985644eb3804911f12c018b88)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:39:15.190886 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e619ae14-5d7b-4950-be32-33dcd1d49865 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:39:15.282951 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e619ae14-5d7b-4950-be32-33dcd1d49865 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7220499985644eb3804911f12c018b88'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:39:15.293332 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e619ae14-5d7b-4950-be32-33dcd1d49865 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7220499985644eb3804911f12c018b88'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:39:15.294601 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e619ae14-5d7b-4950-be32-33dcd1d49865 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7220499985644eb3804911f12c018b88'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:39:15.295750 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e619ae14-5d7b-4950-be32-33dcd1d49865 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7220499985644eb3804911f12c018b88'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:39:15.321535 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e619ae14-5d7b-4950-be32-33dcd1d49865 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7220499985644eb3804911f12c018b88'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:39:15.342017 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1220/2477] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:39:15 2022] DELETE /identity/v3/users/7220499985644eb3804911f12c018b88 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:39:15.643049 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae0066d-bdd1-457e-b5ca-2597b936f890 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:39:15.681532 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae0066d-bdd1-457e-b5ca-2597b936f890 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:39:15.783640 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595fa16d-975b-4102-9ec5-2c3ee3527cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:39:15.785434 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595fa16d-975b-4102-9ec5-2c3ee3527cd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:39:15.786215 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595fa16d-975b-4102-9ec5-2c3ee3527cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:39:15.786902 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595fa16d-975b-4102-9ec5-2c3ee3527cd8 admin admin] PATH_INFO: `/v3/projects/b466c8d09c724e26a0883bd5315593c9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:39:15.789042 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595fa16d-975b-4102-9ec5-2c3ee3527cd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b466c8d09c724e26a0883bd5315593c9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:39:15.791964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595fa16d-975b-4102-9ec5-2c3ee3527cd8 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:39:15.850679 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:39:15.898692 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-595fa16d-975b-4102-9ec5-2c3ee3527cd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b466c8d09c724e26a0883bd5315593c9', 'request_id': 'req-595fa16d-975b-4102-9ec5-2c3ee3527cd8'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:39:15.908681 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-595fa16d-975b-4102-9ec5-2c3ee3527cd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b466c8d09c724e26a0883bd5315593c9', 'request_id': 'req-595fa16d-975b-4102-9ec5-2c3ee3527cd8'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:39:15.930362 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-595fa16d-975b-4102-9ec5-2c3ee3527cd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b466c8d09c724e26a0883bd5315593c9', 'request_id': 'req-595fa16d-975b-4102-9ec5-2c3ee3527cd8'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:39:15.947172 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-595fa16d-975b-4102-9ec5-2c3ee3527cd8 admin admin] The token cache is being invalidate because project b466c8d09c724e26a0883bd5315593c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:39:15.947627 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-595fa16d-975b-4102-9ec5-2c3ee3527cd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:39:15.948616 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-595fa16d-975b-4102-9ec5-2c3ee3527cd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:39:15.951629 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1258/2478] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:39:15 2022] DELETE /identity/v3/projects/b466c8d09c724e26a0883bd5315593c9 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:39:15.955737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e619ae14-5d7b-4950-be32-33dcd1d49865 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:39:15.957484 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e619ae14-5d7b-4950-be32-33dcd1d49865 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:39:15.983681 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e619ae14-5d7b-4950-be32-33dcd1d49865 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:39:15.991233 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:39:16.103341 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b69e2de-2448-49ed-bde4-72583f196713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:39:16.104269 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69e2de-2448-49ed-bde4-72583f196713 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:39:16.104755 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69e2de-2448-49ed-bde4-72583f196713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:39:16.105091 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69e2de-2448-49ed-bde4-72583f196713 admin admin] PATH_INFO: `/v3/users/f09cac1e3a4e4430a055cebab7a7ce9f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:39:16.106989 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b69e2de-2448-49ed-bde4-72583f196713 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f09cac1e3a4e4430a055cebab7a7ce9f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:39:16.109034 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b69e2de-2448-49ed-bde4-72583f196713 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:39:16.180949 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6b69e2de-2448-49ed-bde4-72583f196713 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f09cac1e3a4e4430a055cebab7a7ce9f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:39:16.197103 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6b69e2de-2448-49ed-bde4-72583f196713 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f09cac1e3a4e4430a055cebab7a7ce9f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:39:16.198639 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6b69e2de-2448-49ed-bde4-72583f196713 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f09cac1e3a4e4430a055cebab7a7ce9f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:39:16.199824 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6b69e2de-2448-49ed-bde4-72583f196713 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f09cac1e3a4e4430a055cebab7a7ce9f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:39:16.232327 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6b69e2de-2448-49ed-bde4-72583f196713 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f09cac1e3a4e4430a055cebab7a7ce9f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:39:16.251523 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1221/2479] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:39:15 2022] DELETE /identity/v3/users/f09cac1e3a4e4430a055cebab7a7ce9f => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:39:16.529731 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595fa16d-975b-4102-9ec5-2c3ee3527cd8 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:39:16.532098 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-595fa16d-975b-4102-9ec5-2c3ee3527cd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:39:16.554203 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595fa16d-975b-4102-9ec5-2c3ee3527cd8 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:39:16.629149 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513c6923-5696-4965-a047-1651ba7e3f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:39:16.630331 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513c6923-5696-4965-a047-1651ba7e3f5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:39:16.630670 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513c6923-5696-4965-a047-1651ba7e3f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:39:16.630955 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513c6923-5696-4965-a047-1651ba7e3f5d admin admin] PATH_INFO: `/v3/projects/2cf9cf3dd84b4107b431405d3c2a287b` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:39:16.632379 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513c6923-5696-4965-a047-1651ba7e3f5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cf9cf3dd84b4107b431405d3c2a287b)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:39:16.634712 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513c6923-5696-4965-a047-1651ba7e3f5d admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:39:16.724901 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-513c6923-5696-4965-a047-1651ba7e3f5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cf9cf3dd84b4107b431405d3c2a287b', 'request_id': 'req-513c6923-5696-4965-a047-1651ba7e3f5d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:39:16.735156 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-513c6923-5696-4965-a047-1651ba7e3f5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cf9cf3dd84b4107b431405d3c2a287b', 'request_id': 'req-513c6923-5696-4965-a047-1651ba7e3f5d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:39:16.744987 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-513c6923-5696-4965-a047-1651ba7e3f5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cf9cf3dd84b4107b431405d3c2a287b', 'request_id': 'req-513c6923-5696-4965-a047-1651ba7e3f5d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:39:16.759992 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-513c6923-5696-4965-a047-1651ba7e3f5d admin admin] The token cache is being invalidate because project 2cf9cf3dd84b4107b431405d3c2a287b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:39:16.760448 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-513c6923-5696-4965-a047-1651ba7e3f5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:39:16.761320 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-513c6923-5696-4965-a047-1651ba7e3f5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:39:16.763202 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1259/2480] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:39:16 2022] DELETE /identity/v3/projects/2cf9cf3dd84b4107b431405d3c2a287b => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:39:26.247285 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=50,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:39:26.755490 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=66,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:39:50.442979 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b69e2de-2448-49ed-bde4-72583f196713 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:39:50.444879 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6b69e2de-2448-49ed-bde4-72583f196713 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:39:50.668548 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b69e2de-2448-49ed-bde4-72583f196713 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:39:50.781266 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f69270b-53c7-43f1-863f-e6ec9ff607da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:39:50.782039 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69270b-53c7-43f1-863f-e6ec9ff607da service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:39:50.782283 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69270b-53c7-43f1-863f-e6ec9ff607da service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:39:50.782612 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69270b-53c7-43f1-863f-e6ec9ff607da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:39:50.785424 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8f69270b-53c7-43f1-863f-e6ec9ff607da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:39:50.861220 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:39:50.914010 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f69270b-53c7-43f1-863f-e6ec9ff607da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:39:50.916071 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f69270b-53c7-43f1-863f-e6ec9ff607da service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:39:50.945028 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1222/2481] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:39:50 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:39:52.524665 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513c6923-5696-4965-a047-1651ba7e3f5d admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:39:52.526895 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-513c6923-5696-4965-a047-1651ba7e3f5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:39:52.645979 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513c6923-5696-4965-a047-1651ba7e3f5d admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:39:52.653295 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3dfee8-fd58-494f-a3e0-4f0be53e2935 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:39:52.654359 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3dfee8-fd58-494f-a3e0-4f0be53e2935 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:39:52.654875 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3dfee8-fd58-494f-a3e0-4f0be53e2935 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:39:52.655373 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3dfee8-fd58-494f-a3e0-4f0be53e2935 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:39:52.658553 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-da3dfee8-fd58-494f-a3e0-4f0be53e2935 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:39:52.684424 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3dfee8-fd58-494f-a3e0-4f0be53e2935 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:39:52.687222 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3dfee8-fd58-494f-a3e0-4f0be53e2935 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:39:52.707896 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1260/2482] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:39:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:39:57.311625 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f69270b-53c7-43f1-863f-e6ec9ff607da service nova] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:39:57.313574 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8f69270b-53c7-43f1-863f-e6ec9ff607da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:39:57.395902 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f69270b-53c7-43f1-863f-e6ec9ff607da service nova] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:39:57.401087 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9406bdd6-7f2e-4328-b36b-a282324b6015 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:39:57.401995 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9406bdd6-7f2e-4328-b36b-a282324b6015 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:39:57.402462 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9406bdd6-7f2e-4328-b36b-a282324b6015 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:39:57.402857 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9406bdd6-7f2e-4328-b36b-a282324b6015 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:39:57.404878 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9406bdd6-7f2e-4328-b36b-a282324b6015 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:39:57.415900 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9406bdd6-7f2e-4328-b36b-a282324b6015 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:39:57.417370 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9406bdd6-7f2e-4328-b36b-a282324b6015 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:39:57.430015 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1223/2483] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:39:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:40:02.698934 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=28 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:07.424986 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=45 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:20.453025 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83a3e4-347a-45ec-a3bb-e263efd681d1 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:20.454572 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83a3e4-347a-45ec-a3bb-e263efd681d1 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:20.454572 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83a3e4-347a-45ec-a3bb-e263efd681d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:20.478198 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-6d83a3e4-347a-45ec-a3bb-e263efd681d1 None None] MFA Rules not processed for user `09c9fa7dccb8491c9ad3b43967345ea2`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:40:20.479485 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-6d83a3e4-347a-45ec-a3bb-e263efd681d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:20.578810 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1261/2484] 10.222.0.87 () {58 vars in 1009 bytes} [Tue Jun 21 18:40:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:40:20.644077 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9406bdd6-7f2e-4328-b36b-a282324b6015 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:20.646015 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9406bdd6-7f2e-4328-b36b-a282324b6015 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:20.750170 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9406bdd6-7f2e-4328-b36b-a282324b6015 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:20.758538 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63107cfe-bf12-4186-a8ac-c8b079af3534 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '515b3571045e4fe29565307c8ecc10b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:20.759703 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63107cfe-bf12-4186-a8ac-c8b079af3534 service swift] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:20.760214 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63107cfe-bf12-4186-a8ac-c8b079af3534 service swift] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:20.760679 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63107cfe-bf12-4186-a8ac-c8b079af3534 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:20.773031 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63107cfe-bf12-4186-a8ac-c8b079af3534 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:20.776623 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63107cfe-bf12-4186-a8ac-c8b079af3534 service swift] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:20.793904 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1224/2485] 10.222.0.87 () {60 vars in 1384 bytes} [Tue Jun 21 18:40:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 18:40:24.806382 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d83a3e4-347a-45ec-a3bb-e263efd681d1 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:24.843553 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d83a3e4-347a-45ec-a3bb-e263efd681d1 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:24.850539 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97119903-9e2d-4290-91a9-ff1d211c1507 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:24.851567 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97119903-9e2d-4290-91a9-ff1d211c1507 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:24.851916 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97119903-9e2d-4290-91a9-ff1d211c1507 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:24.852266 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97119903-9e2d-4290-91a9-ff1d211c1507 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:24.855310 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-97119903-9e2d-4290-91a9-ff1d211c1507 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:25.099154 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97119903-9e2d-4290-91a9-ff1d211c1507 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:25.102203 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97119903-9e2d-4290-91a9-ff1d211c1507 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:25.154829 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1262/2486] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:40:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:40:30.787055 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=32 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:31.006832 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63107cfe-bf12-4186-a8ac-c8b079af3534 service swift] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:31.008608 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-63107cfe-bf12-4186-a8ac-c8b079af3534 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:31.031228 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63107cfe-bf12-4186-a8ac-c8b079af3534 service swift] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:31.050751 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a78339-1450-41c6-86b7-4a0f03c4b11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:31.051981 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a78339-1450-41c6-86b7-4a0f03c4b11b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:31.052434 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a78339-1450-41c6-86b7-4a0f03c4b11b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:31.053510 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a78339-1450-41c6-86b7-4a0f03c4b11b admin admin] PATH_INFO: `/v3/users/acf9a33e9fb941939010989a89a51d38` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:31.055230 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a78339-1450-41c6-86b7-4a0f03c4b11b admin admin] RBAC: Authorizing `identity:delete_user(user_id=acf9a33e9fb941939010989a89a51d38)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:31.057732 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a78339-1450-41c6-86b7-4a0f03c4b11b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:31.129303 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c9a78339-1450-41c6-86b7-4a0f03c4b11b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acf9a33e9fb941939010989a89a51d38'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:31.140596 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c9a78339-1450-41c6-86b7-4a0f03c4b11b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acf9a33e9fb941939010989a89a51d38'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:31.141689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c9a78339-1450-41c6-86b7-4a0f03c4b11b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acf9a33e9fb941939010989a89a51d38'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:31.142645 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c9a78339-1450-41c6-86b7-4a0f03c4b11b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acf9a33e9fb941939010989a89a51d38'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:31.170804 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c9a78339-1450-41c6-86b7-4a0f03c4b11b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acf9a33e9fb941939010989a89a51d38'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:31.201979 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1225/2487] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:40:31 2022] DELETE /identity/v3/users/acf9a33e9fb941939010989a89a51d38 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:40:31.503727 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97119903-9e2d-4290-91a9-ff1d211c1507 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:31.532218 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97119903-9e2d-4290-91a9-ff1d211c1507 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:31.700821 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b1b67-2c03-4a3b-a518-6adcc4097238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:31.702405 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b1b67-2c03-4a3b-a518-6adcc4097238 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:31.705051 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b1b67-2c03-4a3b-a518-6adcc4097238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:31.706615 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b1b67-2c03-4a3b-a518-6adcc4097238 admin admin] PATH_INFO: `/v3/projects/3c88935a83d24e408c99cb711ae46257` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:31.709364 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276b1b67-2c03-4a3b-a518-6adcc4097238 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c88935a83d24e408c99cb711ae46257)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:31.714337 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276b1b67-2c03-4a3b-a518-6adcc4097238 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:31.721359 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:31.940172 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-276b1b67-2c03-4a3b-a518-6adcc4097238 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c88935a83d24e408c99cb711ae46257', 'request_id': 'req-276b1b67-2c03-4a3b-a518-6adcc4097238'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:31.951464 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-276b1b67-2c03-4a3b-a518-6adcc4097238 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c88935a83d24e408c99cb711ae46257', 'request_id': 'req-276b1b67-2c03-4a3b-a518-6adcc4097238'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:31.976109 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-276b1b67-2c03-4a3b-a518-6adcc4097238 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c88935a83d24e408c99cb711ae46257', 'request_id': 'req-276b1b67-2c03-4a3b-a518-6adcc4097238'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:32.003920 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-276b1b67-2c03-4a3b-a518-6adcc4097238 admin admin] The token cache is being invalidate because project 3c88935a83d24e408c99cb711ae46257 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:40:32.004412 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-276b1b67-2c03-4a3b-a518-6adcc4097238 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:32.005525 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-276b1b67-2c03-4a3b-a518-6adcc4097238 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:32.008268 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1263/2488] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:40:31 2022] DELETE /identity/v3/projects/3c88935a83d24e408c99cb711ae46257 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:40:32.017671 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cfafee-b945-4af6-a2ff-9db40b8d11ef None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:32.018267 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cfafee-b945-4af6-a2ff-9db40b8d11ef None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:32.018816 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cfafee-b945-4af6-a2ff-9db40b8d11ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:32.149784 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f1cfafee-b945-4af6-a2ff-9db40b8d11ef None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:40:32.152309 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f1cfafee-b945-4af6-a2ff-9db40b8d11ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:32.271005 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1226/2489] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:40:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:40:32.274428 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f32ff-e66f-4e03-aeef-db92acfa5274 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:32.275112 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f32ff-e66f-4e03-aeef-db92acfa5274 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:32.276634 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f32ff-e66f-4e03-aeef-db92acfa5274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:32.325604 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-a24f32ff-e66f-4e03-aeef-db92acfa5274 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:40:32.328629 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a24f32ff-e66f-4e03-aeef-db92acfa5274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:32.345466 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1264/2490] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:40:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:40:32.351108 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cfafee-b945-4af6-a2ff-9db40b8d11ef None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:32.365739 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cfafee-b945-4af6-a2ff-9db40b8d11ef None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:32.372398 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b7ac37-c977-41db-80c8-1837f384fed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:32.373160 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7ac37-c977-41db-80c8-1837f384fed8 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:32.373364 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7ac37-c977-41db-80c8-1837f384fed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:32.373945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7ac37-c977-41db-80c8-1837f384fed8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:32.374652 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b7ac37-c977-41db-80c8-1837f384fed8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:40:32.375198 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b7ac37-c977-41db-80c8-1837f384fed8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:32.378476 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b7ac37-c977-41db-80c8-1837f384fed8 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:32.403340 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1227/2491] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:40:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:40:32.409256 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24f32ff-e66f-4e03-aeef-db92acfa5274 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:32.428151 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24f32ff-e66f-4e03-aeef-db92acfa5274 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:32.438149 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962e669-23dc-410c-b0a1-27793c2cd57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:32.439435 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962e669-23dc-410c-b0a1-27793c2cd57d admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:32.439922 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962e669-23dc-410c-b0a1-27793c2cd57d admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:32.440406 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962e669-23dc-410c-b0a1-27793c2cd57d admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:32.442089 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1962e669-23dc-410c-b0a1-27793c2cd57d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:32.446183 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1962e669-23dc-410c-b0a1-27793c2cd57d admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:32.492616 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1265/2492] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:40:32 2022] POST /identity/v3/projects => generated 371 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:40:32.497956 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b7ac37-c977-41db-80c8-1837f384fed8 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:32.513967 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b7ac37-c977-41db-80c8-1837f384fed8 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:32.587990 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:32.647124 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fd5bd8-a9fa-4fe9-844e-4c82cd8a9962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:32.647923 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fd5bd8-a9fa-4fe9-844e-4c82cd8a9962 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:32.648272 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fd5bd8-a9fa-4fe9-844e-4c82cd8a9962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:32.648629 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fd5bd8-a9fa-4fe9-844e-4c82cd8a9962 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:32.649695 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fd5bd8-a9fa-4fe9-844e-4c82cd8a9962 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:32.651798 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fd5bd8-a9fa-4fe9-844e-4c82cd8a9962 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:32.682737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1228/2493] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:40:32 2022] POST /identity/v3/users => generated 286 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:40:32.689562 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962e669-23dc-410c-b0a1-27793c2cd57d admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:32.709843 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962e669-23dc-410c-b0a1-27793c2cd57d admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:32.721973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d86a3f1-c390-407f-81d6-ee8bfce443cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:32.724066 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86a3f1-c390-407f-81d6-ee8bfce443cf admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:32.724383 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86a3f1-c390-407f-81d6-ee8bfce443cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:32.724908 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86a3f1-c390-407f-81d6-ee8bfce443cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:32.726408 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d86a3f1-c390-407f-81d6-ee8bfce443cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:32.729754 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d86a3f1-c390-407f-81d6-ee8bfce443cf admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:32.748866 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1266/2494] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:40:32 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:40:32.755353 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fd5bd8-a9fa-4fe9-844e-4c82cd8a9962 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:32.771879 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fd5bd8-a9fa-4fe9-844e-4c82cd8a9962 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:32.780853 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7701599a-2c04-4c09-9c95-2151b42f54ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:32.782176 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7701599a-2c04-4c09-9c95-2151b42f54ea admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:32.782695 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7701599a-2c04-4c09-9c95-2151b42f54ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:32.783431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7701599a-2c04-4c09-9c95-2151b42f54ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:32.784883 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7701599a-2c04-4c09-9c95-2151b42f54ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:32.788589 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7701599a-2c04-4c09-9c95-2151b42f54ea admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:32.817526 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d86a3f1-c390-407f-81d6-ee8bfce443cf admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:32.818629 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1229/2495] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:40:32 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:40:32.834631 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d86a3f1-c390-407f-81d6-ee8bfce443cf admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:32.851858 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb813f1d-0314-45d1-af9d-e9439cb0753f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:32.853398 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb813f1d-0314-45d1-af9d-e9439cb0753f admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:32.853914 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb813f1d-0314-45d1-af9d-e9439cb0753f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:32.854330 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb813f1d-0314-45d1-af9d-e9439cb0753f admin admin] PATH_INFO: `/v3/projects/aa108438fb6d4660975eb3589e79d97c/users/ded133a0c82942409998864f5e589c6a/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:32.883636 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb813f1d-0314-45d1-af9d-e9439cb0753f admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa108438fb6d4660975eb3589e79d97c, user_id=ded133a0c82942409998864f5e589c6a, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:32.887974 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb813f1d-0314-45d1-af9d-e9439cb0753f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:32.923638 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1267/2496] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:40:32 2022] PUT /identity/v3/projects/aa108438fb6d4660975eb3589e79d97c/users/ded133a0c82942409998864f5e589c6a/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:40:32.943683 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d137ace-26bc-4aab-8ae3-a752592941e5 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:32.944235 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d137ace-26bc-4aab-8ae3-a752592941e5 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:32.944235 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d137ace-26bc-4aab-8ae3-a752592941e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:32.998185 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-2d137ace-26bc-4aab-8ae3-a752592941e5 None None] MFA Rules not processed for user `ded133a0c82942409998864f5e589c6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:40:33.000006 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2d137ace-26bc-4aab-8ae3-a752592941e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:33.149125 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1230/2497] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:40:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:40:33.168479 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb813f1d-0314-45d1-af9d-e9439cb0753f admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:33.170432 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-bb813f1d-0314-45d1-af9d-e9439cb0753f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:33.340260 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb813f1d-0314-45d1-af9d-e9439cb0753f admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:33.362495 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a57ee9-0912-4d48-b466-333196911e9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:33.362495 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a57ee9-0912-4d48-b466-333196911e9d service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:33.362495 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a57ee9-0912-4d48-b466-333196911e9d service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:33.362495 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a57ee9-0912-4d48-b466-333196911e9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:33.377302 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a57ee9-0912-4d48-b466-333196911e9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:33.380737 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a57ee9-0912-4d48-b466-333196911e9d service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:33.403948 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1268/2498] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:40:33 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:40:36.348862 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d137ace-26bc-4aab-8ae3-a752592941e5 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:36.434212 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d137ace-26bc-4aab-8ae3-a752592941e5 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:36.439768 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763952cf-8808-4f58-9464-2e218bb58f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:36.440744 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763952cf-8808-4f58-9464-2e218bb58f35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:36.441075 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763952cf-8808-4f58-9464-2e218bb58f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:36.441515 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763952cf-8808-4f58-9464-2e218bb58f35 admin admin] PATH_INFO: `/v3/users/ded133a0c82942409998864f5e589c6a` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:36.443077 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763952cf-8808-4f58-9464-2e218bb58f35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ded133a0c82942409998864f5e589c6a)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:36.445158 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763952cf-8808-4f58-9464-2e218bb58f35 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:36.498407 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:36.507187 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-763952cf-8808-4f58-9464-2e218bb58f35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ded133a0c82942409998864f5e589c6a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:36.521394 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-763952cf-8808-4f58-9464-2e218bb58f35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ded133a0c82942409998864f5e589c6a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:36.522217 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-763952cf-8808-4f58-9464-2e218bb58f35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ded133a0c82942409998864f5e589c6a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:36.522837 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-763952cf-8808-4f58-9464-2e218bb58f35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ded133a0c82942409998864f5e589c6a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:36.541072 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-763952cf-8808-4f58-9464-2e218bb58f35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ded133a0c82942409998864f5e589c6a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:36.560791 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1231/2499] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:40:36 2022] DELETE /identity/v3/users/ded133a0c82942409998864f5e589c6a => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:40:36.566431 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e14aa2-3aba-4c0e-b7e8-09f518cd3c1c None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:36.567053 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e14aa2-3aba-4c0e-b7e8-09f518cd3c1c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:36.567192 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e14aa2-3aba-4c0e-b7e8-09f518cd3c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:36.586900 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-f5e14aa2-3aba-4c0e-b7e8-09f518cd3c1c None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:40:36.588335 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f5e14aa2-3aba-4c0e-b7e8-09f518cd3c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:36.599724 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:36.685547 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1269/2500] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:40:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:40:36.702483 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763952cf-8808-4f58-9464-2e218bb58f35 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:36.704727 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-763952cf-8808-4f58-9464-2e218bb58f35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:36.807459 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763952cf-8808-4f58-9464-2e218bb58f35 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:36.811705 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e9faf-e60c-4e8c-b23d-b8860bad101b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:36.812325 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e9faf-e60c-4e8c-b23d-b8860bad101b service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:36.812618 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e9faf-e60c-4e8c-b23d-b8860bad101b service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:36.812903 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e9faf-e60c-4e8c-b23d-b8860bad101b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:36.824148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8e9faf-e60c-4e8c-b23d-b8860bad101b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:36.825772 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8e9faf-e60c-4e8c-b23d-b8860bad101b service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:36.838085 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1232/2501] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:40:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:40:37.357333 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e14aa2-3aba-4c0e-b7e8-09f518cd3c1c None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:37.360102 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f5e14aa2-3aba-4c0e-b7e8-09f518cd3c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:37.386405 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e14aa2-3aba-4c0e-b7e8-09f518cd3c1c None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:37.399237 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91344973-1cc9-4fdf-b614-79500beb89ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:37.400737 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91344973-1cc9-4fdf-b614-79500beb89ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:37.401201 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91344973-1cc9-4fdf-b614-79500beb89ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:37.401725 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91344973-1cc9-4fdf-b614-79500beb89ae admin admin] PATH_INFO: `/v3/projects/aa108438fb6d4660975eb3589e79d97c` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:37.404455 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91344973-1cc9-4fdf-b614-79500beb89ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa108438fb6d4660975eb3589e79d97c)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:37.407977 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91344973-1cc9-4fdf-b614-79500beb89ae admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:37.486466 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-91344973-1cc9-4fdf-b614-79500beb89ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa108438fb6d4660975eb3589e79d97c', 'request_id': 'req-91344973-1cc9-4fdf-b614-79500beb89ae'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:37.497189 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-91344973-1cc9-4fdf-b614-79500beb89ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa108438fb6d4660975eb3589e79d97c', 'request_id': 'req-91344973-1cc9-4fdf-b614-79500beb89ae'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:37.510459 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-91344973-1cc9-4fdf-b614-79500beb89ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa108438fb6d4660975eb3589e79d97c', 'request_id': 'req-91344973-1cc9-4fdf-b614-79500beb89ae'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:37.531745 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-91344973-1cc9-4fdf-b614-79500beb89ae admin admin] The token cache is being invalidate because project aa108438fb6d4660975eb3589e79d97c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:40:37.532130 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-91344973-1cc9-4fdf-b614-79500beb89ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:37.533196 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-91344973-1cc9-4fdf-b614-79500beb89ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:37.537985 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1270/2502] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:40:37 2022] DELETE /identity/v3/projects/aa108438fb6d4660975eb3589e79d97c => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:40:37.542032 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e8cff-f06f-4abc-a5b2-ce9f9b836c7c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:37.542659 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e8cff-f06f-4abc-a5b2-ce9f9b836c7c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:37.542803 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e8cff-f06f-4abc-a5b2-ce9f9b836c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:37.569070 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-337e8cff-f06f-4abc-a5b2-ce9f9b836c7c None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:40:37.570775 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-337e8cff-f06f-4abc-a5b2-ce9f9b836c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:37.687366 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1233/2503] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:40:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:40:37.701395 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61211d5-7f79-48b7-8246-bae18f0883e5 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:37.702369 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61211d5-7f79-48b7-8246-bae18f0883e5 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:37.702369 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61211d5-7f79-48b7-8246-bae18f0883e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:37.741440 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-c61211d5-7f79-48b7-8246-bae18f0883e5 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:40:37.743473 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c61211d5-7f79-48b7-8246-bae18f0883e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:37.759384 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1271/2504] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:40:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:40:37.767352 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337e8cff-f06f-4abc-a5b2-ce9f9b836c7c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:37.787016 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337e8cff-f06f-4abc-a5b2-ce9f9b836c7c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:37.796020 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2c189c-06c8-4f1d-a4ac-121252c74c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:37.797432 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c189c-06c8-4f1d-a4ac-121252c74c1d admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:37.798094 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c189c-06c8-4f1d-a4ac-121252c74c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:37.798588 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c189c-06c8-4f1d-a4ac-121252c74c1d admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:37.800085 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2c189c-06c8-4f1d-a4ac-121252c74c1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:40:37.800644 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2c189c-06c8-4f1d-a4ac-121252c74c1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:37.803842 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2c189c-06c8-4f1d-a4ac-121252c74c1d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:37.835907 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1234/2505] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:40:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:40:37.846648 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61211d5-7f79-48b7-8246-bae18f0883e5 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:37.866335 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61211d5-7f79-48b7-8246-bae18f0883e5 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:37.878426 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427f67f8-dc59-49ec-922d-ebed5131224c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:37.879777 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f67f8-dc59-49ec-922d-ebed5131224c admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:37.880259 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f67f8-dc59-49ec-922d-ebed5131224c admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:37.880778 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f67f8-dc59-49ec-922d-ebed5131224c admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:37.882355 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427f67f8-dc59-49ec-922d-ebed5131224c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:37.887726 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427f67f8-dc59-49ec-922d-ebed5131224c admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:37.947492 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1272/2506] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:40:37 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:40:37.952184 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2c189c-06c8-4f1d-a4ac-121252c74c1d admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:37.968134 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2c189c-06c8-4f1d-a4ac-121252c74c1d admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:38.034964 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:38.079533 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34f7660-932f-4422-aff3-c21573cf525a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:38.080702 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f7660-932f-4422-aff3-c21573cf525a admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:38.081206 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f7660-932f-4422-aff3-c21573cf525a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:38.081765 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f7660-932f-4422-aff3-c21573cf525a admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:38.082614 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34f7660-932f-4422-aff3-c21573cf525a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:38.084701 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34f7660-932f-4422-aff3-c21573cf525a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:38.115289 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1235/2507] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:40:37 2022] POST /identity/v3/users => generated 287 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:40:38.120263 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427f67f8-dc59-49ec-922d-ebed5131224c admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:38.139162 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427f67f8-dc59-49ec-922d-ebed5131224c admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:38.146737 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8eb483-0dc0-4333-86a3-26196b57a276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:38.147857 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8eb483-0dc0-4333-86a3-26196b57a276 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:38.148430 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8eb483-0dc0-4333-86a3-26196b57a276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:38.148908 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8eb483-0dc0-4333-86a3-26196b57a276 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:38.149959 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8eb483-0dc0-4333-86a3-26196b57a276 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:38.152679 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8eb483-0dc0-4333-86a3-26196b57a276 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:38.175318 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1273/2508] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:40:38.181591 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34f7660-932f-4422-aff3-c21573cf525a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:38.206269 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34f7660-932f-4422-aff3-c21573cf525a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:38.222875 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da142ea-5e12-4e3c-b452-98f0b42cdaa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:38.223655 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da142ea-5e12-4e3c-b452-98f0b42cdaa1 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:38.224443 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da142ea-5e12-4e3c-b452-98f0b42cdaa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:38.225000 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da142ea-5e12-4e3c-b452-98f0b42cdaa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:38.226523 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da142ea-5e12-4e3c-b452-98f0b42cdaa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:38.228748 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da142ea-5e12-4e3c-b452-98f0b42cdaa1 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:38.245989 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1236/2509] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:40:38.253895 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8eb483-0dc0-4333-86a3-26196b57a276 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:38.268656 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8eb483-0dc0-4333-86a3-26196b57a276 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:38.277496 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714fa31c-4695-4f91-9fd3-5cce6a984237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:38.279001 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714fa31c-4695-4f91-9fd3-5cce6a984237 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:38.279445 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714fa31c-4695-4f91-9fd3-5cce6a984237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:38.279979 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714fa31c-4695-4f91-9fd3-5cce6a984237 admin admin] PATH_INFO: `/v3/projects/891e26050f10474b9f70bc46d5b1975d/users/187d5f81ce704deab74fb20bbe23bb0d/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:38.310150 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714fa31c-4695-4f91-9fd3-5cce6a984237 admin admin] RBAC: Authorizing `identity:create_grant(project_id=891e26050f10474b9f70bc46d5b1975d, user_id=187d5f81ce704deab74fb20bbe23bb0d, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:38.315779 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714fa31c-4695-4f91-9fd3-5cce6a984237 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:38.336476 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1274/2510] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:40:38 2022] PUT /identity/v3/projects/891e26050f10474b9f70bc46d5b1975d/users/187d5f81ce704deab74fb20bbe23bb0d/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:40:38.348408 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b576981-932b-409c-8a9b-89f2faa2c475 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:38.348733 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b576981-932b-409c-8a9b-89f2faa2c475 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:38.348998 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b576981-932b-409c-8a9b-89f2faa2c475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:38.384331 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-0b576981-932b-409c-8a9b-89f2faa2c475 None None] MFA Rules not processed for user `187d5f81ce704deab74fb20bbe23bb0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:40:38.385810 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0b576981-932b-409c-8a9b-89f2faa2c475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:38.468065 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1237/2511] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:40:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:40:38.496894 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714fa31c-4695-4f91-9fd3-5cce6a984237 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:38.499365 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-714fa31c-4695-4f91-9fd3-5cce6a984237 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:38.548164 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:38.604137 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714fa31c-4695-4f91-9fd3-5cce6a984237 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:38.611158 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fc9e7f-d158-4016-a1e9-994ac3c0809e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:38.612020 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc9e7f-d158-4016-a1e9-994ac3c0809e service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:38.612382 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc9e7f-d158-4016-a1e9-994ac3c0809e service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:38.612754 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc9e7f-d158-4016-a1e9-994ac3c0809e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:38.623020 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fc9e7f-d158-4016-a1e9-994ac3c0809e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:38.625074 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fc9e7f-d158-4016-a1e9-994ac3c0809e service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:38.637829 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1275/2512] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:40:38 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:40:38.947631 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b576981-932b-409c-8a9b-89f2faa2c475 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:39.040457 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b576981-932b-409c-8a9b-89f2faa2c475 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:39.046058 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b871986a-6bc5-400e-8aa5-b3db10840c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:39.047056 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b871986a-6bc5-400e-8aa5-b3db10840c24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:39.047346 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b871986a-6bc5-400e-8aa5-b3db10840c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:39.047622 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b871986a-6bc5-400e-8aa5-b3db10840c24 admin admin] PATH_INFO: `/v3/users/187d5f81ce704deab74fb20bbe23bb0d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:39.049261 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b871986a-6bc5-400e-8aa5-b3db10840c24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=187d5f81ce704deab74fb20bbe23bb0d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:39.051369 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b871986a-6bc5-400e-8aa5-b3db10840c24 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:39.128254 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:39.131523 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b871986a-6bc5-400e-8aa5-b3db10840c24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '187d5f81ce704deab74fb20bbe23bb0d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:39.145473 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b871986a-6bc5-400e-8aa5-b3db10840c24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '187d5f81ce704deab74fb20bbe23bb0d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:39.146677 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b871986a-6bc5-400e-8aa5-b3db10840c24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '187d5f81ce704deab74fb20bbe23bb0d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:39.147585 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b871986a-6bc5-400e-8aa5-b3db10840c24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '187d5f81ce704deab74fb20bbe23bb0d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:39.164025 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b871986a-6bc5-400e-8aa5-b3db10840c24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '187d5f81ce704deab74fb20bbe23bb0d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:39.182034 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1238/2513] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:40:38 2022] DELETE /identity/v3/users/187d5f81ce704deab74fb20bbe23bb0d => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:40:39.188137 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9817b9-3ba7-4e54-ad61-44b0ce040acc None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:39.189013 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9817b9-3ba7-4e54-ad61-44b0ce040acc None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:39.189173 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9817b9-3ba7-4e54-ad61-44b0ce040acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:39.220302 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-0c9817b9-3ba7-4e54-ad61-44b0ce040acc None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:40:39.222656 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-0c9817b9-3ba7-4e54-ad61-44b0ce040acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:39.311516 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e16d3b-32f3-4442-8e83-f83e7abc57d0 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:39.312036 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e16d3b-32f3-4442-8e83-f83e7abc57d0 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:39.314430 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e16d3b-32f3-4442-8e83-f83e7abc57d0 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:39.317588 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1239/2514] 10.222.0.87 () {54 vars in 921 bytes} [Tue Jun 21 18:40:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:40:39.327473 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a57af5-3fb7-4db8-b0c0-5bbc72035a8c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:39.328488 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a57af5-3fb7-4db8-b0c0-5bbc72035a8c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:39.329045 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a57af5-3fb7-4db8-b0c0-5bbc72035a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:39.361257 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b1a57af5-3fb7-4db8-b0c0-5bbc72035a8c None None] MFA Rules not processed for user `a1b3fa67905c4027b3656ac36e2b76c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:40:39.363373 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b1a57af5-3fb7-4db8-b0c0-5bbc72035a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:39.365912 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1276/2515] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:40:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:40:39.382165 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9817b9-3ba7-4e54-ad61-44b0ce040acc None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:39.385223 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-0c9817b9-3ba7-4e54-ad61-44b0ce040acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:39.492220 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1240/2516] 10.222.0.87 () {58 vars in 1018 bytes} [Tue Jun 21 18:40:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:40:39.494991 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9817b9-3ba7-4e54-ad61-44b0ce040acc None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:39.502894 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39063dc5-555f-4b46-bca6-0adbc16edae5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:39.503625 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39063dc5-555f-4b46-bca6-0adbc16edae5 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:39.504053 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39063dc5-555f-4b46-bca6-0adbc16edae5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:39.504433 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39063dc5-555f-4b46-bca6-0adbc16edae5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:39.509824 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a57af5-3fb7-4db8-b0c0-5bbc72035a8c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:39.512031 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b1a57af5-3fb7-4db8-b0c0-5bbc72035a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:39.523586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39063dc5-555f-4b46-bca6-0adbc16edae5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:39.526719 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39063dc5-555f-4b46-bca6-0adbc16edae5 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:39.549944 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1277/2517] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:40:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:40:39.606552 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a57af5-3fb7-4db8-b0c0-5bbc72035a8c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:39.611204 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f6a14b-2436-4354-8366-3203c978afb5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7da4bf6f3be47b4aa25c6240cb4be03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:39.611972 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6a14b-2436-4354-8366-3203c978afb5 service glance] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:39.612263 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6a14b-2436-4354-8366-3203c978afb5 service glance] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:39.612738 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6a14b-2436-4354-8366-3203c978afb5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:39.625759 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f6a14b-2436-4354-8366-3203c978afb5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:39.627579 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f6a14b-2436-4354-8366-3203c978afb5 service glance] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:39.642238 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1241/2518] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:40:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:40:39.747491 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38205277-536a-44b9-be01-4090851f3204 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:39.747770 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38205277-536a-44b9-be01-4090851f3204 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:39.748193 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38205277-536a-44b9-be01-4090851f3204 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:39.749873 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1278/2519] 10.222.0.87 () {54 vars in 921 bytes} [Tue Jun 21 18:40:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:40:39.757742 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b3399-a1b0-4161-b840-9823d7532c27 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:39.758237 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b3399-a1b0-4161-b840-9823d7532c27 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:39.758690 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b3399-a1b0-4161-b840-9823d7532c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:39.825137 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-e17b3399-a1b0-4161-b840-9823d7532c27 None None] MFA Rules not processed for user `480e2529015c4c2fb4d6cd1985009fde`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:40:39.827349 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e17b3399-a1b0-4161-b840-9823d7532c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:39.840180 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1242/2520] 10.222.0.87 () {58 vars in 1018 bytes} [Tue Jun 21 18:40:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:40:39.857160 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38205277-536a-44b9-be01-4090851f3204 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:39.860003 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-38205277-536a-44b9-be01-4090851f3204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:39.881451 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38205277-536a-44b9-be01-4090851f3204 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:39.886721 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2708ae9-d49d-4a9f-864c-a36f9cf7a2a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:39.887742 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2708ae9-d49d-4a9f-864c-a36f9cf7a2a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:39.888154 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2708ae9-d49d-4a9f-864c-a36f9cf7a2a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:39.888591 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2708ae9-d49d-4a9f-864c-a36f9cf7a2a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:39.898955 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2708ae9-d49d-4a9f-864c-a36f9cf7a2a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:39.900430 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2708ae9-d49d-4a9f-864c-a36f9cf7a2a4 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:39.912590 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1279/2521] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:40:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:40:39.962195 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17b3399-a1b0-4161-b840-9823d7532c27 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:39.965666 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e17b3399-a1b0-4161-b840-9823d7532c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:39.985423 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17b3399-a1b0-4161-b840-9823d7532c27 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:39.996323 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b4170e-9f3a-4ca8-9c75-49f2ca9a76db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:39.998202 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4170e-9f3a-4ca8-9c75-49f2ca9a76db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:39.998541 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4170e-9f3a-4ca8-9c75-49f2ca9a76db admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:40.000017 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4170e-9f3a-4ca8-9c75-49f2ca9a76db admin admin] PATH_INFO: `/v3/projects/891e26050f10474b9f70bc46d5b1975d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:40.011780 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b4170e-9f3a-4ca8-9c75-49f2ca9a76db admin admin] RBAC: Authorizing `identity:delete_project(project_id=891e26050f10474b9f70bc46d5b1975d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:40.014447 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b4170e-9f3a-4ca8-9c75-49f2ca9a76db admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:40.062731 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:40.086937 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-49b4170e-9f3a-4ca8-9c75-49f2ca9a76db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '891e26050f10474b9f70bc46d5b1975d', 'request_id': 'req-49b4170e-9f3a-4ca8-9c75-49f2ca9a76db'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:40.099097 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-49b4170e-9f3a-4ca8-9c75-49f2ca9a76db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '891e26050f10474b9f70bc46d5b1975d', 'request_id': 'req-49b4170e-9f3a-4ca8-9c75-49f2ca9a76db'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:40.106521 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-49b4170e-9f3a-4ca8-9c75-49f2ca9a76db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '891e26050f10474b9f70bc46d5b1975d', 'request_id': 'req-49b4170e-9f3a-4ca8-9c75-49f2ca9a76db'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:40.122081 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-49b4170e-9f3a-4ca8-9c75-49f2ca9a76db admin admin] The token cache is being invalidate because project 891e26050f10474b9f70bc46d5b1975d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:40:40.122367 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-49b4170e-9f3a-4ca8-9c75-49f2ca9a76db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:40.123272 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-49b4170e-9f3a-4ca8-9c75-49f2ca9a76db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:40.125175 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1243/2522] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:40:39 2022] DELETE /identity/v3/projects/891e26050f10474b9f70bc46d5b1975d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:40:40.131314 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ce023-03b2-4834-80f4-4c39c641833d None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:40.131657 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ce023-03b2-4834-80f4-4c39c641833d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:40.131919 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ce023-03b2-4834-80f4-4c39c641833d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:40.155402 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-205ce023-03b2-4834-80f4-4c39c641833d None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:40:40.158821 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-205ce023-03b2-4834-80f4-4c39c641833d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:40.218149 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:40.237133 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1280/2523] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:40:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:40:40.248795 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8afb7c8-f3e2-4109-95d5-61c4b6034204 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:40.249086 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8afb7c8-f3e2-4109-95d5-61c4b6034204 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:40.249422 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8afb7c8-f3e2-4109-95d5-61c4b6034204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:40.271459 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d8afb7c8-f3e2-4109-95d5-61c4b6034204 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:40:40.272779 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d8afb7c8-f3e2-4109-95d5-61c4b6034204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:40.281617 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1244/2524] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:40:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:40:40.286709 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ce023-03b2-4834-80f4-4c39c641833d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:40.298069 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ce023-03b2-4834-80f4-4c39c641833d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:40.303550 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc6a03-d8d9-4e7a-8cf2-0eab0fa5e923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:40.304263 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc6a03-d8d9-4e7a-8cf2-0eab0fa5e923 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:40.304548 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc6a03-d8d9-4e7a-8cf2-0eab0fa5e923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:40.304830 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc6a03-d8d9-4e7a-8cf2-0eab0fa5e923 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:40.305588 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebc6a03-d8d9-4e7a-8cf2-0eab0fa5e923 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:40:40.306107 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebc6a03-d8d9-4e7a-8cf2-0eab0fa5e923 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:40.307800 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebc6a03-d8d9-4e7a-8cf2-0eab0fa5e923 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:40.332268 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1281/2525] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:40:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:40:40.338428 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8afb7c8-f3e2-4109-95d5-61c4b6034204 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:40.355585 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8afb7c8-f3e2-4109-95d5-61c4b6034204 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:40.364095 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d780347d-7568-4942-a3e7-6df0778a9860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:40.365322 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780347d-7568-4942-a3e7-6df0778a9860 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:40.365884 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780347d-7568-4942-a3e7-6df0778a9860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:40.366379 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780347d-7568-4942-a3e7-6df0778a9860 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:40.367602 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d780347d-7568-4942-a3e7-6df0778a9860 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:40.371601 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d780347d-7568-4942-a3e7-6df0778a9860 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:40.423297 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1245/2526] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:40:40 2022] POST /identity/v3/projects => generated 405 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:40:40.429538 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc6a03-d8d9-4e7a-8cf2-0eab0fa5e923 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:40.447903 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc6a03-d8d9-4e7a-8cf2-0eab0fa5e923 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:40.550373 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3287e5f2-fc4e-460c-9896-51ec9358bdf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:40.551125 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287e5f2-fc4e-460c-9896-51ec9358bdf8 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:40.551517 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287e5f2-fc4e-460c-9896-51ec9358bdf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:40.551887 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287e5f2-fc4e-460c-9896-51ec9358bdf8 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:40.552700 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3287e5f2-fc4e-460c-9896-51ec9358bdf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:40.555131 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3287e5f2-fc4e-460c-9896-51ec9358bdf8 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:40.595058 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1282/2527] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:40:40 2022] POST /identity/v3/users => generated 303 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:40:40.599905 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d780347d-7568-4942-a3e7-6df0778a9860 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:40.615117 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d780347d-7568-4942-a3e7-6df0778a9860 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:40.620610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e9296e-9e15-4490-bbfc-7e994678a66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:40.621486 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9296e-9e15-4490-bbfc-7e994678a66a admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:40.621820 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9296e-9e15-4490-bbfc-7e994678a66a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:40.622109 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9296e-9e15-4490-bbfc-7e994678a66a admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:40.622879 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e9296e-9e15-4490-bbfc-7e994678a66a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:40.624609 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e9296e-9e15-4490-bbfc-7e994678a66a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:40.638442 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1246/2528] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:40:40 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:40:40.643542 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3287e5f2-fc4e-460c-9896-51ec9358bdf8 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:40.659818 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3287e5f2-fc4e-460c-9896-51ec9358bdf8 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:40.665852 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e526f1f-9bbc-415d-91a8-f3044533dbc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:40.666724 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e526f1f-9bbc-415d-91a8-f3044533dbc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:40.667131 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e526f1f-9bbc-415d-91a8-f3044533dbc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:40.667528 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e526f1f-9bbc-415d-91a8-f3044533dbc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:40.668520 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e526f1f-9bbc-415d-91a8-f3044533dbc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:40.671195 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e526f1f-9bbc-415d-91a8-f3044533dbc6 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:40.685416 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1283/2529] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:40:40 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:40:40.693209 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e9296e-9e15-4490-bbfc-7e994678a66a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:40.713190 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e9296e-9e15-4490-bbfc-7e994678a66a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:40.726172 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b4dcc0-e77e-40d0-aa50-67c8d5839d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:40.727398 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4dcc0-e77e-40d0-aa50-67c8d5839d2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:40.727740 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4dcc0-e77e-40d0-aa50-67c8d5839d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:40.728078 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4dcc0-e77e-40d0-aa50-67c8d5839d2b admin admin] PATH_INFO: `/v3/projects/851eec387d894f12832641bbe37d35b3/users/d0977ae439a5423d9c8a046087606310/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:40.753805 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b4dcc0-e77e-40d0-aa50-67c8d5839d2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=851eec387d894f12832641bbe37d35b3, user_id=d0977ae439a5423d9c8a046087606310, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:40.757144 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b4dcc0-e77e-40d0-aa50-67c8d5839d2b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:40.777061 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1247/2530] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:40:40 2022] PUT /identity/v3/projects/851eec387d894f12832641bbe37d35b3/users/d0977ae439a5423d9c8a046087606310/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:40:40.783840 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7521508e-2876-4986-a0fa-32b1d61bd84e None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:40.784205 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7521508e-2876-4986-a0fa-32b1d61bd84e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:40.784542 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7521508e-2876-4986-a0fa-32b1d61bd84e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:40.804906 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-7521508e-2876-4986-a0fa-32b1d61bd84e None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:40:40.806117 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7521508e-2876-4986-a0fa-32b1d61bd84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:40.914021 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1284/2531] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:40:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:40:40.928086 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b4dcc0-e77e-40d0-aa50-67c8d5839d2b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:40.930380 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b0b4dcc0-e77e-40d0-aa50-67c8d5839d2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:41.015487 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b4dcc0-e77e-40d0-aa50-67c8d5839d2b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:41.023458 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb113e6e-5165-44d9-999c-b5359d141c58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:41.024674 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb113e6e-5165-44d9-999c-b5359d141c58 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:41.025334 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb113e6e-5165-44d9-999c-b5359d141c58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:41.026302 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb113e6e-5165-44d9-999c-b5359d141c58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:41.040278 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb113e6e-5165-44d9-999c-b5359d141c58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:41.042507 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb113e6e-5165-44d9-999c-b5359d141c58 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:41.048091 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9a12e-97ee-4beb-9098-34e284a79ab4 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:41.048652 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9a12e-97ee-4beb-9098-34e284a79ab4 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:41.049059 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9a12e-97ee-4beb-9098-34e284a79ab4 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:41.051124 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1285/2532] 10.222.0.87 () {54 vars in 921 bytes} [Tue Jun 21 18:40:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:40:41.055012 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1248/2533] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:40:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:40:41.058478 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c454d122-47da-4d38-b0a1-d554787f85ba None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:41.058886 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c454d122-47da-4d38-b0a1-d554787f85ba None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:41.059258 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c454d122-47da-4d38-b0a1-d554787f85ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:41.080884 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-c454d122-47da-4d38-b0a1-d554787f85ba None None] MFA Rules not processed for user `a1b3fa67905c4027b3656ac36e2b76c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:40:41.082726 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c454d122-47da-4d38-b0a1-d554787f85ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:41.150875 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:41.180892 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1286/2534] 10.222.0.87 () {58 vars in 1018 bytes} [Tue Jun 21 18:40:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:40:41.193451 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb113e6e-5165-44d9-999c-b5359d141c58 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:41.195669 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-eb113e6e-5165-44d9-999c-b5359d141c58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:41.213511 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb113e6e-5165-44d9-999c-b5359d141c58 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:41.219835 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68298313-0492-4b4c-bf0d-b319ffd9c2fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:41.220767 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68298313-0492-4b4c-bf0d-b319ffd9c2fb service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:41.221331 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68298313-0492-4b4c-bf0d-b319ffd9c2fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:41.221870 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68298313-0492-4b4c-bf0d-b319ffd9c2fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:41.241410 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68298313-0492-4b4c-bf0d-b319ffd9c2fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:41.244342 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68298313-0492-4b4c-bf0d-b319ffd9c2fb service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:41.263711 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1249/2535] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:40:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:40:41.352046 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c454d122-47da-4d38-b0a1-d554787f85ba None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:41.353711 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c454d122-47da-4d38-b0a1-d554787f85ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:41.464509 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c454d122-47da-4d38-b0a1-d554787f85ba None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:41.472025 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919479f6-e3ad-41c3-8f4f-b8933d3ce7c1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef1d49b36cdc4665b82487be97e9dcd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:41.473486 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919479f6-e3ad-41c3-8f4f-b8933d3ce7c1 service placement] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:41.473486 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919479f6-e3ad-41c3-8f4f-b8933d3ce7c1 service placement] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:41.473486 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919479f6-e3ad-41c3-8f4f-b8933d3ce7c1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:41.475852 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-919479f6-e3ad-41c3-8f4f-b8933d3ce7c1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:41.492040 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919479f6-e3ad-41c3-8f4f-b8933d3ce7c1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:41.494023 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919479f6-e3ad-41c3-8f4f-b8933d3ce7c1 service placement] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:41.507092 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1287/2536] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:40:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:40:41.909968 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e317f-e6c4-4b2d-8f3f-eae353308ec9 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:41.910347 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e317f-e6c4-4b2d-8f3f-eae353308ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:41.910811 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e317f-e6c4-4b2d-8f3f-eae353308ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:41.949979 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-e29e317f-e6c4-4b2d-8f3f-eae353308ec9 None None] MFA Rules not processed for user `d0977ae439a5423d9c8a046087606310`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:40:41.951351 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e29e317f-e6c4-4b2d-8f3f-eae353308ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:41.996428 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:42.043180 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1250/2537] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:40:41 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:40:42.048285 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919479f6-e3ad-41c3-8f4f-b8933d3ce7c1 service placement] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:42.063560 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919479f6-e3ad-41c3-8f4f-b8933d3ce7c1 service placement] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:42.072443 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9344bc4-6a65-43c4-a5cc-304866873161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:42.073629 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9344bc4-6a65-43c4-a5cc-304866873161 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:42.074187 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9344bc4-6a65-43c4-a5cc-304866873161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:42.074725 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9344bc4-6a65-43c4-a5cc-304866873161 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:42.076427 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9344bc4-6a65-43c4-a5cc-304866873161 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:42.081784 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9344bc4-6a65-43c4-a5cc-304866873161 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:42.129138 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1288/2538] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:40:42 2022] POST /identity/v3/projects => generated 405 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:40:42.133073 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29e317f-e6c4-4b2d-8f3f-eae353308ec9 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:42.134845 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9344bc4-6a65-43c4-a5cc-304866873161 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:42.135556 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e29e317f-e6c4-4b2d-8f3f-eae353308ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:42.161041 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29e317f-e6c4-4b2d-8f3f-eae353308ec9 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:42.245792 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9344bc4-6a65-43c4-a5cc-304866873161 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:42.250990 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603eb2e-3256-4c2a-bea4-903685297ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:42.251739 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603eb2e-3256-4c2a-bea4-903685297ddc admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:42.252179 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603eb2e-3256-4c2a-bea4-903685297ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:42.252466 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603eb2e-3256-4c2a-bea4-903685297ddc admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:42.253178 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a603eb2e-3256-4c2a-bea4-903685297ddc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:42.255380 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a603eb2e-3256-4c2a-bea4-903685297ddc admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:42.266764 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe8b02-872d-4e48-aa5a-4fa25fbb52eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:42.267799 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe8b02-872d-4e48-aa5a-4fa25fbb52eb service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:42.268145 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe8b02-872d-4e48-aa5a-4fa25fbb52eb service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:42.268528 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe8b02-872d-4e48-aa5a-4fa25fbb52eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:42.272877 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d3fe8b02-872d-4e48-aa5a-4fa25fbb52eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:42.286950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1289/2539] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:40:42 2022] POST /identity/v3/users => generated 309 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:40:42.288885 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603eb2e-3256-4c2a-bea4-903685297ddc admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:42.308281 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603eb2e-3256-4c2a-bea4-903685297ddc admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:42.317146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51bcd8d-cd5f-468d-960d-18f4678052d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:42.318384 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51bcd8d-cd5f-468d-960d-18f4678052d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:42.318836 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51bcd8d-cd5f-468d-960d-18f4678052d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:42.319340 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51bcd8d-cd5f-468d-960d-18f4678052d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:42.320716 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51bcd8d-cd5f-468d-960d-18f4678052d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:42.323762 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51bcd8d-cd5f-468d-960d-18f4678052d9 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:42.345670 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1290/2540] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:40:42.350294 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51bcd8d-cd5f-468d-960d-18f4678052d9 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:42.366908 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51bcd8d-cd5f-468d-960d-18f4678052d9 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:42.373405 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e300b0c-d4dc-4a94-a5d1-1bacebbc22f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:42.374399 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e300b0c-d4dc-4a94-a5d1-1bacebbc22f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:42.374877 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e300b0c-d4dc-4a94-a5d1-1bacebbc22f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:42.375182 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e300b0c-d4dc-4a94-a5d1-1bacebbc22f5 admin admin] PATH_INFO: `/v3/projects/7e5b029c6c1446a295f0f6ccbcc7c71c/users/f9a24cd17c7947388fcf69edb1418cde/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:42.381429 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fe8b02-872d-4e48-aa5a-4fa25fbb52eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:42.383233 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fe8b02-872d-4e48-aa5a-4fa25fbb52eb service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:42.403750 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e300b0c-d4dc-4a94-a5d1-1bacebbc22f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e5b029c6c1446a295f0f6ccbcc7c71c, user_id=f9a24cd17c7947388fcf69edb1418cde, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:42.405057 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1251/2541] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:40:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:40:42.407453 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e300b0c-d4dc-4a94-a5d1-1bacebbc22f5 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:42.426671 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1291/2542] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:40:42 2022] PUT /identity/v3/projects/7e5b029c6c1446a295f0f6ccbcc7c71c/users/f9a24cd17c7947388fcf69edb1418cde/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:40:42.430285 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe8b02-872d-4e48-aa5a-4fa25fbb52eb service nova] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:42.441737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe8b02-872d-4e48-aa5a-4fa25fbb52eb service nova] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:42.499439 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:42.528091 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed8102-f22e-45f9-95c9-f437071c1642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:42.529192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed8102-f22e-45f9-95c9-f437071c1642 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:42.529688 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed8102-f22e-45f9-95c9-f437071c1642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:42.530191 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed8102-f22e-45f9-95c9-f437071c1642 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:42.531558 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ed8102-f22e-45f9-95c9-f437071c1642 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:42.534688 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ed8102-f22e-45f9-95c9-f437071c1642 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:42.556216 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1252/2543] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:40:42.558098 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e300b0c-d4dc-4a94-a5d1-1bacebbc22f5 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:42.571788 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e300b0c-d4dc-4a94-a5d1-1bacebbc22f5 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:42.578794 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0973f8a-97b8-48e3-92d2-c67c865a29fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:42.580211 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0973f8a-97b8-48e3-92d2-c67c865a29fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:42.580556 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0973f8a-97b8-48e3-92d2-c67c865a29fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:42.580876 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0973f8a-97b8-48e3-92d2-c67c865a29fe admin admin] PATH_INFO: `/v3/projects/7e5b029c6c1446a295f0f6ccbcc7c71c/users/f9a24cd17c7947388fcf69edb1418cde/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:42.583572 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0973f8a-97b8-48e3-92d2-c67c865a29fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e5b029c6c1446a295f0f6ccbcc7c71c, user_id=f9a24cd17c7947388fcf69edb1418cde, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:42.586969 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0973f8a-97b8-48e3-92d2-c67c865a29fe admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:42.590952 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:42.638838 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1292/2544] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:40:42 2022] PUT /identity/v3/projects/7e5b029c6c1446a295f0f6ccbcc7c71c/users/f9a24cd17c7947388fcf69edb1418cde/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:40:43.609417 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab845a7b-71c7-447d-9c8f-79d1a9ba628b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:43.610226 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab845a7b-71c7-447d-9c8f-79d1a9ba628b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:43.610427 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab845a7b-71c7-447d-9c8f-79d1a9ba628b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:43.660393 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-ab845a7b-71c7-447d-9c8f-79d1a9ba628b None None] MFA Rules not processed for user `f9a24cd17c7947388fcf69edb1418cde`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:40:43.662547 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ab845a7b-71c7-447d-9c8f-79d1a9ba628b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:43.762400 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1253/2545] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:40:43 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:40:45.256989 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0973f8a-97b8-48e3-92d2-c67c865a29fe admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:45.340384 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0973f8a-97b8-48e3-92d2-c67c865a29fe admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:45.346173 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9721754-31af-48bf-b228-82a6d21b76ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:45.347373 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9721754-31af-48bf-b228-82a6d21b76ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:45.347373 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9721754-31af-48bf-b228-82a6d21b76ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:45.347681 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9721754-31af-48bf-b228-82a6d21b76ef admin admin] PATH_INFO: `/v3/users/d0977ae439a5423d9c8a046087606310` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:45.348979 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9721754-31af-48bf-b228-82a6d21b76ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0977ae439a5423d9c8a046087606310)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:45.351186 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9721754-31af-48bf-b228-82a6d21b76ef admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:45.407247 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-b9721754-31af-48bf-b228-82a6d21b76ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0977ae439a5423d9c8a046087606310'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:45.424896 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-b9721754-31af-48bf-b228-82a6d21b76ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0977ae439a5423d9c8a046087606310'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:45.425720 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-b9721754-31af-48bf-b228-82a6d21b76ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0977ae439a5423d9c8a046087606310'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:45.426364 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-b9721754-31af-48bf-b228-82a6d21b76ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0977ae439a5423d9c8a046087606310'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:45.435110 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-b9721754-31af-48bf-b228-82a6d21b76ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0977ae439a5423d9c8a046087606310'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:45.455324 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1293/2546] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:40:45 2022] DELETE /identity/v3/users/d0977ae439a5423d9c8a046087606310 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:40:46.050200 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab845a7b-71c7-447d-9c8f-79d1a9ba628b None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:46.064390 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab845a7b-71c7-447d-9c8f-79d1a9ba628b None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:46.138539 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd82340-79d2-46d0-989d-44d88d61bce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:46.140138 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd82340-79d2-46d0-989d-44d88d61bce0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:46.140641 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd82340-79d2-46d0-989d-44d88d61bce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:46.141131 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd82340-79d2-46d0-989d-44d88d61bce0 admin admin] PATH_INFO: `/v3/projects/851eec387d894f12832641bbe37d35b3` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:46.143420 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd82340-79d2-46d0-989d-44d88d61bce0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=851eec387d894f12832641bbe37d35b3)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:46.147420 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd82340-79d2-46d0-989d-44d88d61bce0 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:46.220658 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ddd82340-79d2-46d0-989d-44d88d61bce0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '851eec387d894f12832641bbe37d35b3', 'request_id': 'req-ddd82340-79d2-46d0-989d-44d88d61bce0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:46.232625 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ddd82340-79d2-46d0-989d-44d88d61bce0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '851eec387d894f12832641bbe37d35b3', 'request_id': 'req-ddd82340-79d2-46d0-989d-44d88d61bce0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:46.240336 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ddd82340-79d2-46d0-989d-44d88d61bce0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '851eec387d894f12832641bbe37d35b3', 'request_id': 'req-ddd82340-79d2-46d0-989d-44d88d61bce0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:46.242663 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:46.257999 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ddd82340-79d2-46d0-989d-44d88d61bce0 admin admin] The token cache is being invalidate because project 851eec387d894f12832641bbe37d35b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:40:46.258405 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ddd82340-79d2-46d0-989d-44d88d61bce0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:46.259619 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ddd82340-79d2-46d0-989d-44d88d61bce0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:46.261610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1254/2547] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:40:46 2022] DELETE /identity/v3/projects/851eec387d894f12832641bbe37d35b3 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:40:46.265907 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9721754-31af-48bf-b228-82a6d21b76ef admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:46.267658 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b9721754-31af-48bf-b228-82a6d21b76ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:46.281372 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9721754-31af-48bf-b228-82a6d21b76ef admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:46.347433 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5492132a-3d04-4eb7-942f-3cf3dda3e904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:46.348543 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5492132a-3d04-4eb7-942f-3cf3dda3e904 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:46.348805 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5492132a-3d04-4eb7-942f-3cf3dda3e904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:46.349084 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5492132a-3d04-4eb7-942f-3cf3dda3e904 admin admin] PATH_INFO: `/v3/users/f9a24cd17c7947388fcf69edb1418cde` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:46.350416 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5492132a-3d04-4eb7-942f-3cf3dda3e904 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9a24cd17c7947388fcf69edb1418cde)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:46.352408 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5492132a-3d04-4eb7-942f-3cf3dda3e904 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:46.375502 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:46.414720 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5492132a-3d04-4eb7-942f-3cf3dda3e904 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9a24cd17c7947388fcf69edb1418cde'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:46.430696 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5492132a-3d04-4eb7-942f-3cf3dda3e904 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9a24cd17c7947388fcf69edb1418cde'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:46.431492 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5492132a-3d04-4eb7-942f-3cf3dda3e904 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9a24cd17c7947388fcf69edb1418cde'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:46.432137 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5492132a-3d04-4eb7-942f-3cf3dda3e904 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9a24cd17c7947388fcf69edb1418cde'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:46.441212 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5492132a-3d04-4eb7-942f-3cf3dda3e904 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9a24cd17c7947388fcf69edb1418cde'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:46.456854 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1294/2548] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:40:46 2022] DELETE /identity/v3/users/f9a24cd17c7947388fcf69edb1418cde => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:40:46.694556 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd82340-79d2-46d0-989d-44d88d61bce0 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:46.696117 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ddd82340-79d2-46d0-989d-44d88d61bce0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:46.714335 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd82340-79d2-46d0-989d-44d88d61bce0 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:46.789716 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1569ac2c-454c-4329-af0d-03ac1823c50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:46.790561 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1569ac2c-454c-4329-af0d-03ac1823c50e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:46.790839 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1569ac2c-454c-4329-af0d-03ac1823c50e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:46.791117 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1569ac2c-454c-4329-af0d-03ac1823c50e admin admin] PATH_INFO: `/v3/projects/7e5b029c6c1446a295f0f6ccbcc7c71c` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:46.792475 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1569ac2c-454c-4329-af0d-03ac1823c50e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e5b029c6c1446a295f0f6ccbcc7c71c)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:46.794493 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1569ac2c-454c-4329-af0d-03ac1823c50e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:46.851685 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-1569ac2c-454c-4329-af0d-03ac1823c50e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e5b029c6c1446a295f0f6ccbcc7c71c', 'request_id': 'req-1569ac2c-454c-4329-af0d-03ac1823c50e'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:46.860723 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-1569ac2c-454c-4329-af0d-03ac1823c50e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e5b029c6c1446a295f0f6ccbcc7c71c', 'request_id': 'req-1569ac2c-454c-4329-af0d-03ac1823c50e'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:46.867902 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-1569ac2c-454c-4329-af0d-03ac1823c50e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e5b029c6c1446a295f0f6ccbcc7c71c', 'request_id': 'req-1569ac2c-454c-4329-af0d-03ac1823c50e'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:46.881509 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-1569ac2c-454c-4329-af0d-03ac1823c50e admin admin] The token cache is being invalidate because project 7e5b029c6c1446a295f0f6ccbcc7c71c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:40:46.881836 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-1569ac2c-454c-4329-af0d-03ac1823c50e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:46.882557 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-1569ac2c-454c-4329-af0d-03ac1823c50e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:46.884182 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1255/2549] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:40:46 2022] DELETE /identity/v3/projects/7e5b029c6c1446a295f0f6ccbcc7c71c => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:40:46.889489 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5a7b3-fb50-4600-8d8c-897873ca4532 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:46.889847 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5a7b3-fb50-4600-8d8c-897873ca4532 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:46.890141 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5a7b3-fb50-4600-8d8c-897873ca4532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:46.917521 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-3db5a7b3-fb50-4600-8d8c-897873ca4532 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:40:46.918642 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3db5a7b3-fb50-4600-8d8c-897873ca4532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:46.988170 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1295/2550] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:40:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:40:46.997736 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacface4-c685-4f44-a31f-4314e0dc51d6 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:46.998126 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacface4-c685-4f44-a31f-4314e0dc51d6 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:46.998410 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacface4-c685-4f44-a31f-4314e0dc51d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:47.021072 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-eacface4-c685-4f44-a31f-4314e0dc51d6 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:40:47.022345 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-eacface4-c685-4f44-a31f-4314e0dc51d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:47.032767 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1256/2551] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:40:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:40:47.037600 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db5a7b3-fb50-4600-8d8c-897873ca4532 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:47.052450 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db5a7b3-fb50-4600-8d8c-897873ca4532 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:47.058809 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2519fe25-0069-4d3b-a6d5-754dd2a15c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:47.059509 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519fe25-0069-4d3b-a6d5-754dd2a15c82 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:47.059799 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519fe25-0069-4d3b-a6d5-754dd2a15c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:47.060063 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519fe25-0069-4d3b-a6d5-754dd2a15c82 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:47.060960 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2519fe25-0069-4d3b-a6d5-754dd2a15c82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:40:47.061267 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2519fe25-0069-4d3b-a6d5-754dd2a15c82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:47.063037 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2519fe25-0069-4d3b-a6d5-754dd2a15c82 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:47.083619 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1296/2552] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:40:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:40:47.088780 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacface4-c685-4f44-a31f-4314e0dc51d6 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:47.100079 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacface4-c685-4f44-a31f-4314e0dc51d6 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:47.107833 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b63a3-351b-42b4-b97b-02d03a76dd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:47.108617 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b63a3-351b-42b4-b97b-02d03a76dd92 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:47.108815 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b63a3-351b-42b4-b97b-02d03a76dd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:47.109105 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b63a3-351b-42b4-b97b-02d03a76dd92 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:47.109861 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038b63a3-351b-42b4-b97b-02d03a76dd92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:47.112250 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038b63a3-351b-42b4-b97b-02d03a76dd92 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:47.149672 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1257/2553] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:40:47 2022] POST /identity/v3/projects => generated 365 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:40:47.155146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2519fe25-0069-4d3b-a6d5-754dd2a15c82 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:47.170043 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2519fe25-0069-4d3b-a6d5-754dd2a15c82 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:47.259243 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae059312-23c2-41c6-bacb-80d0600fe78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:47.259943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae059312-23c2-41c6-bacb-80d0600fe78d admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:47.260228 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae059312-23c2-41c6-bacb-80d0600fe78d admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:47.260501 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae059312-23c2-41c6-bacb-80d0600fe78d admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:47.261212 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae059312-23c2-41c6-bacb-80d0600fe78d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:47.263363 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae059312-23c2-41c6-bacb-80d0600fe78d admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:47.288060 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1297/2554] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:40:47 2022] POST /identity/v3/users => generated 283 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:40:47.292855 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b63a3-351b-42b4-b97b-02d03a76dd92 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:47.304753 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b63a3-351b-42b4-b97b-02d03a76dd92 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:47.310936 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bcbe29-2d11-4ec0-a568-bee304115194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:47.311823 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bcbe29-2d11-4ec0-a568-bee304115194 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:47.312216 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bcbe29-2d11-4ec0-a568-bee304115194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:47.312505 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bcbe29-2d11-4ec0-a568-bee304115194 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:47.313312 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bcbe29-2d11-4ec0-a568-bee304115194 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:47.315419 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bcbe29-2d11-4ec0-a568-bee304115194 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:47.331398 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1258/2555] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:40:47 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:40:47.336392 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae059312-23c2-41c6-bacb-80d0600fe78d admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:47.347838 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae059312-23c2-41c6-bacb-80d0600fe78d admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:47.353679 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b13bc48-594f-4df6-b28f-999334fe6c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:47.354521 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b13bc48-594f-4df6-b28f-999334fe6c35 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:47.354886 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b13bc48-594f-4df6-b28f-999334fe6c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:47.355267 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b13bc48-594f-4df6-b28f-999334fe6c35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:47.356465 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b13bc48-594f-4df6-b28f-999334fe6c35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:47.358407 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b13bc48-594f-4df6-b28f-999334fe6c35 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:47.365576 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:47.373002 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1298/2556] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:40:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:40:47.378304 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bcbe29-2d11-4ec0-a568-bee304115194 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:47.396585 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bcbe29-2d11-4ec0-a568-bee304115194 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:47.402957 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344fca6a-4998-4e8d-a79d-976989170250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:47.403946 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344fca6a-4998-4e8d-a79d-976989170250 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:47.404235 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344fca6a-4998-4e8d-a79d-976989170250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:47.404518 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344fca6a-4998-4e8d-a79d-976989170250 admin admin] PATH_INFO: `/v3/projects/c50377ed21554b97a4aa399831e9b435/users/a42c1aae23754c0c9cdafe4ddd7d0558/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:47.421872 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344fca6a-4998-4e8d-a79d-976989170250 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c50377ed21554b97a4aa399831e9b435, user_id=a42c1aae23754c0c9cdafe4ddd7d0558, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:47.424628 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344fca6a-4998-4e8d-a79d-976989170250 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:47.439749 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1259/2557] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:40:47 2022] PUT /identity/v3/projects/c50377ed21554b97a4aa399831e9b435/users/a42c1aae23754c0c9cdafe4ddd7d0558/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:40:47.450275 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6afa4-fa7f-4f07-a8b5-d0db9d4b2c29 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:47.450563 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6afa4-fa7f-4f07-a8b5-d0db9d4b2c29 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:47.450850 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6afa4-fa7f-4f07-a8b5-d0db9d4b2c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:47.481658 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-b0f6afa4-fa7f-4f07-a8b5-d0db9d4b2c29 None None] MFA Rules not processed for user `a42c1aae23754c0c9cdafe4ddd7d0558`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:40:47.482943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b0f6afa4-fa7f-4f07-a8b5-d0db9d4b2c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:47.559976 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1299/2558] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:40:47 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:40:47.565135 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344fca6a-4998-4e8d-a79d-976989170250 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:47.596503 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:47.645854 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344fca6a-4998-4e8d-a79d-976989170250 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:47.651445 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d98f0b-3209-45f6-8933-2c34392ef61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:47.652223 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d98f0b-3209-45f6-8933-2c34392ef61c admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:47.652537 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d98f0b-3209-45f6-8933-2c34392ef61c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:47.652832 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d98f0b-3209-45f6-8933-2c34392ef61c admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:47.653569 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d98f0b-3209-45f6-8933-2c34392ef61c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:47.655672 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d98f0b-3209-45f6-8933-2c34392ef61c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:47.692091 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1260/2559] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:40:47 2022] POST /identity/v3/projects => generated 365 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:40:47.698422 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f6afa4-fa7f-4f07-a8b5-d0db9d4b2c29 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:47.718158 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f6afa4-fa7f-4f07-a8b5-d0db9d4b2c29 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:47.811881 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2c1fb-2ade-444a-80f1-24688fd257b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:47.812563 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2c1fb-2ade-444a-80f1-24688fd257b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:47.812853 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2c1fb-2ade-444a-80f1-24688fd257b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:47.813130 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2c1fb-2ade-444a-80f1-24688fd257b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:47.814026 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f2c1fb-2ade-444a-80f1-24688fd257b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:47.816148 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f2c1fb-2ade-444a-80f1-24688fd257b2 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:47.844985 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1300/2560] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:40:47 2022] POST /identity/v3/users => generated 289 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:40:47.850727 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d98f0b-3209-45f6-8933-2c34392ef61c admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:47.862683 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d98f0b-3209-45f6-8933-2c34392ef61c admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:47.868693 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3b59f-28d6-4926-b47b-e70a4dd07cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:47.869394 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3b59f-28d6-4926-b47b-e70a4dd07cc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:47.869714 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3b59f-28d6-4926-b47b-e70a4dd07cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:47.869985 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3b59f-28d6-4926-b47b-e70a4dd07cc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:47.870751 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd3b59f-28d6-4926-b47b-e70a4dd07cc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:47.872462 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd3b59f-28d6-4926-b47b-e70a4dd07cc1 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:47.886321 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1261/2561] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:40:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:40:47.890996 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2c1fb-2ade-444a-80f1-24688fd257b2 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:47.903410 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2c1fb-2ade-444a-80f1-24688fd257b2 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:47.909060 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd1bb24-608a-4c34-923b-186d320cbc6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:47.910029 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd1bb24-608a-4c34-923b-186d320cbc6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:47.910317 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd1bb24-608a-4c34-923b-186d320cbc6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:47.910804 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd1bb24-608a-4c34-923b-186d320cbc6a admin admin] PATH_INFO: `/v3/projects/168bfacc9fb549129401b8ed7fd59b87/users/99bbe5b59d4247d8806b2db931f06338/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:47.929921 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd1bb24-608a-4c34-923b-186d320cbc6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=168bfacc9fb549129401b8ed7fd59b87, user_id=99bbe5b59d4247d8806b2db931f06338, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:47.933107 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd1bb24-608a-4c34-923b-186d320cbc6a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:47.952349 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1301/2562] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:40:47 2022] PUT /identity/v3/projects/168bfacc9fb549129401b8ed7fd59b87/users/99bbe5b59d4247d8806b2db931f06338/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:40:47.957108 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3b59f-28d6-4926-b47b-e70a4dd07cc1 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:47.971094 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3b59f-28d6-4926-b47b-e70a4dd07cc1 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:48.047000 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11ea477-aa27-4e71-b453-f00df6cc6a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:48.047740 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ea477-aa27-4e71-b453-f00df6cc6a44 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:48.048104 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ea477-aa27-4e71-b453-f00df6cc6a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:48.048394 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ea477-aa27-4e71-b453-f00df6cc6a44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:48.049213 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11ea477-aa27-4e71-b453-f00df6cc6a44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:48.051071 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11ea477-aa27-4e71-b453-f00df6cc6a44 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:48.064057 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1262/2563] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:40:47 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:40:48.069127 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd1bb24-608a-4c34-923b-186d320cbc6a admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:48.088567 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd1bb24-608a-4c34-923b-186d320cbc6a admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:48.097567 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec9dbb2-ceda-46e9-879f-bf8b8b471aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:48.099188 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9dbb2-ceda-46e9-879f-bf8b8b471aa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:48.099698 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9dbb2-ceda-46e9-879f-bf8b8b471aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:48.100213 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9dbb2-ceda-46e9-879f-bf8b8b471aa8 admin admin] PATH_INFO: `/v3/projects/168bfacc9fb549129401b8ed7fd59b87/users/99bbe5b59d4247d8806b2db931f06338/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:48.103903 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec9dbb2-ceda-46e9-879f-bf8b8b471aa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=168bfacc9fb549129401b8ed7fd59b87, user_id=99bbe5b59d4247d8806b2db931f06338, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:48.109404 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec9dbb2-ceda-46e9-879f-bf8b8b471aa8 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:48.123440 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1302/2564] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:40:48 2022] PUT /identity/v3/projects/168bfacc9fb549129401b8ed7fd59b87/users/99bbe5b59d4247d8806b2db931f06338/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:40:48.135799 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64a774-aeb3-49ab-95b7-19a121eae3c2 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:48.136535 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64a774-aeb3-49ab-95b7-19a121eae3c2 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:48.136535 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64a774-aeb3-49ab-95b7-19a121eae3c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:48.168056 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-2e64a774-aeb3-49ab-95b7-19a121eae3c2 None None] MFA Rules not processed for user `99bbe5b59d4247d8806b2db931f06338`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:40:48.169294 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2e64a774-aeb3-49ab-95b7-19a121eae3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:48.254531 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1263/2565] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:40:48 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:40:48.270428 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec9dbb2-ceda-46e9-879f-bf8b8b471aa8 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:48.272850 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-bec9dbb2-ceda-46e9-879f-bf8b8b471aa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:48.370073 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:48.422142 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec9dbb2-ceda-46e9-879f-bf8b8b471aa8 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:48.429780 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe239f3-8096-4569-a461-2a6d3469d1ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:48.430929 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe239f3-8096-4569-a461-2a6d3469d1ee service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:48.431460 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe239f3-8096-4569-a461-2a6d3469d1ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:48.431969 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe239f3-8096-4569-a461-2a6d3469d1ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:48.450893 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe239f3-8096-4569-a461-2a6d3469d1ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:48.454173 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe239f3-8096-4569-a461-2a6d3469d1ee service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:48.474956 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1303/2566] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:40:48 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:40:48.591961 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e64a774-aeb3-49ab-95b7-19a121eae3c2 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:48.604001 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e64a774-aeb3-49ab-95b7-19a121eae3c2 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:48.608363 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7da8bea-5b9e-449f-9f7e-2416b4b3f094 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:48.609515 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da8bea-5b9e-449f-9f7e-2416b4b3f094 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:48.610050 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da8bea-5b9e-449f-9f7e-2416b4b3f094 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:48.610558 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da8bea-5b9e-449f-9f7e-2416b4b3f094 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:48.636618 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:48.732097 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7da8bea-5b9e-449f-9f7e-2416b4b3f094 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:48.733760 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7da8bea-5b9e-449f-9f7e-2416b4b3f094 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:48.746872 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1264/2567] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:40:48 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:40:57.321962 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230751c5-f322-4ace-a917-4a9fd72a7abd None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:57.322737 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230751c5-f322-4ace-a917-4a9fd72a7abd None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:57.322737 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230751c5-f322-4ace-a917-4a9fd72a7abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:57.358353 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-230751c5-f322-4ace-a917-4a9fd72a7abd None None] MFA Rules not processed for user `09c9fa7dccb8491c9ad3b43967345ea2`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:40:57.359606 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-230751c5-f322-4ace-a917-4a9fd72a7abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:57.449770 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1304/2568] 10.222.0.87 () {58 vars in 1009 bytes} [Tue Jun 21 18:40:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:40:57.464989 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7da8bea-5b9e-449f-9f7e-2416b4b3f094 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:57.466601 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e7da8bea-5b9e-449f-9f7e-2416b4b3f094 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:57.581717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7da8bea-5b9e-449f-9f7e-2416b4b3f094 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:57.588184 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f725ed3-3b92-48a9-a164-d2f99ed4c03a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '515b3571045e4fe29565307c8ecc10b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:57.588995 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f725ed3-3b92-48a9-a164-d2f99ed4c03a service swift] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:57.589188 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f725ed3-3b92-48a9-a164-d2f99ed4c03a service swift] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:57.589543 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f725ed3-3b92-48a9-a164-d2f99ed4c03a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:57.603867 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f725ed3-3b92-48a9-a164-d2f99ed4c03a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:57.605536 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f725ed3-3b92-48a9-a164-d2f99ed4c03a service swift] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:57.615740 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1265/2569] 10.222.0.87 () {60 vars in 1384 bytes} [Tue Jun 21 18:40:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 18:41:01.332355 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230751c5-f322-4ace-a917-4a9fd72a7abd None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:01.522009 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230751c5-f322-4ace-a917-4a9fd72a7abd None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:01.524929 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe930c-a6d6-4df8-a4d3-f6bfd96046b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:01.527805 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe930c-a6d6-4df8-a4d3-f6bfd96046b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:01.528305 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe930c-a6d6-4df8-a4d3-f6bfd96046b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:01.528920 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe930c-a6d6-4df8-a4d3-f6bfd96046b0 admin admin] PATH_INFO: `/v3/users/a42c1aae23754c0c9cdafe4ddd7d0558` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:01.531690 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfe930c-a6d6-4df8-a4d3-f6bfd96046b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a42c1aae23754c0c9cdafe4ddd7d0558)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:01.536154 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfe930c-a6d6-4df8-a4d3-f6bfd96046b0 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:01.680567 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-bcfe930c-a6d6-4df8-a4d3-f6bfd96046b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a42c1aae23754c0c9cdafe4ddd7d0558'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:01.700206 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:41:01.715309 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-bcfe930c-a6d6-4df8-a4d3-f6bfd96046b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a42c1aae23754c0c9cdafe4ddd7d0558'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:01.716525 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-bcfe930c-a6d6-4df8-a4d3-f6bfd96046b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a42c1aae23754c0c9cdafe4ddd7d0558'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:01.718215 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-bcfe930c-a6d6-4df8-a4d3-f6bfd96046b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a42c1aae23754c0c9cdafe4ddd7d0558'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:01.737720 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-bcfe930c-a6d6-4df8-a4d3-f6bfd96046b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a42c1aae23754c0c9cdafe4ddd7d0558'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:01.766774 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1305/2570] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:41:01 2022] DELETE /identity/v3/users/a42c1aae23754c0c9cdafe4ddd7d0558 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:41:01.768321 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae7ba8-3128-453f-aeeb-3bbbb64258b7 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:01.769128 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae7ba8-3128-453f-aeeb-3bbbb64258b7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:01.769318 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae7ba8-3128-453f-aeeb-3bbbb64258b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:01.803545 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a9ae7ba8-3128-453f-aeeb-3bbbb64258b7 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:41:01.805835 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a9ae7ba8-3128-453f-aeeb-3bbbb64258b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:01.959837 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1266/2571] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:41:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:41:01.977338 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe930c-a6d6-4df8-a4d3-f6bfd96046b0 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:01.980366 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-bcfe930c-a6d6-4df8-a4d3-f6bfd96046b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:02.100147 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe930c-a6d6-4df8-a4d3-f6bfd96046b0 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:02.106466 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3b955d-49fd-4704-92a9-f334bbe89ad3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:02.107452 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b955d-49fd-4704-92a9-f334bbe89ad3 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:02.107794 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b955d-49fd-4704-92a9-f334bbe89ad3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:02.108248 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b955d-49fd-4704-92a9-f334bbe89ad3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:02.125802 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3b955d-49fd-4704-92a9-f334bbe89ad3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:02.128770 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3b955d-49fd-4704-92a9-f334bbe89ad3 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:02.145603 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1306/2572] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:41:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:41:02.641480 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ae7ba8-3128-453f-aeeb-3bbbb64258b7 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:02.644813 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a9ae7ba8-3128-453f-aeeb-3bbbb64258b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:02.718525 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ae7ba8-3128-453f-aeeb-3bbbb64258b7 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:02.727468 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39518edf-c192-4f30-a399-ce56bdb306b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:02.728416 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39518edf-c192-4f30-a399-ce56bdb306b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:02.728700 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39518edf-c192-4f30-a399-ce56bdb306b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:02.728977 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39518edf-c192-4f30-a399-ce56bdb306b3 admin admin] PATH_INFO: `/v3/projects/c50377ed21554b97a4aa399831e9b435` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:02.731554 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39518edf-c192-4f30-a399-ce56bdb306b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c50377ed21554b97a4aa399831e9b435)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:02.758011 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39518edf-c192-4f30-a399-ce56bdb306b3 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:02.919506 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:41:02.935372 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-39518edf-c192-4f30-a399-ce56bdb306b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c50377ed21554b97a4aa399831e9b435', 'request_id': 'req-39518edf-c192-4f30-a399-ce56bdb306b3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:02.955502 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-39518edf-c192-4f30-a399-ce56bdb306b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c50377ed21554b97a4aa399831e9b435', 'request_id': 'req-39518edf-c192-4f30-a399-ce56bdb306b3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:02.968805 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-39518edf-c192-4f30-a399-ce56bdb306b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c50377ed21554b97a4aa399831e9b435', 'request_id': 'req-39518edf-c192-4f30-a399-ce56bdb306b3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:03.000146 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-39518edf-c192-4f30-a399-ce56bdb306b3 admin admin] The token cache is being invalidate because project c50377ed21554b97a4aa399831e9b435 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:41:03.000630 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-39518edf-c192-4f30-a399-ce56bdb306b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:03.005666 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-39518edf-c192-4f30-a399-ce56bdb306b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:03.006231 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1267/2573] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:41:02 2022] DELETE /identity/v3/projects/c50377ed21554b97a4aa399831e9b435 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:41:03.010709 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3b955d-49fd-4704-92a9-f334bbe89ad3 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:03.032467 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3b955d-49fd-4704-92a9-f334bbe89ad3 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:03.174620 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bff1a6-4786-450d-8a8e-297251b81561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:03.176460 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bff1a6-4786-450d-8a8e-297251b81561 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:03.176947 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bff1a6-4786-450d-8a8e-297251b81561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:03.177422 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bff1a6-4786-450d-8a8e-297251b81561 admin admin] PATH_INFO: `/v3/users/99bbe5b59d4247d8806b2db931f06338` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:03.179862 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bff1a6-4786-450d-8a8e-297251b81561 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99bbe5b59d4247d8806b2db931f06338)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:03.183726 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bff1a6-4786-450d-8a8e-297251b81561 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:03.268887 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-96bff1a6-4786-450d-8a8e-297251b81561 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99bbe5b59d4247d8806b2db931f06338'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:03.291906 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-96bff1a6-4786-450d-8a8e-297251b81561 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99bbe5b59d4247d8806b2db931f06338'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:03.293103 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-96bff1a6-4786-450d-8a8e-297251b81561 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99bbe5b59d4247d8806b2db931f06338'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:03.293843 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-96bff1a6-4786-450d-8a8e-297251b81561 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99bbe5b59d4247d8806b2db931f06338'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:03.306044 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-96bff1a6-4786-450d-8a8e-297251b81561 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99bbe5b59d4247d8806b2db931f06338'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:03.315554 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:41:03.340665 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1307/2574] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:41:03 2022] DELETE /identity/v3/users/99bbe5b59d4247d8806b2db931f06338 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:41:03.781778 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39518edf-c192-4f30-a399-ce56bdb306b3 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:03.784119 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-39518edf-c192-4f30-a399-ce56bdb306b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:03.807480 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39518edf-c192-4f30-a399-ce56bdb306b3 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:03.921405 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e069972c-cb18-4497-b8a7-4db2eaf42e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:03.922499 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e069972c-cb18-4497-b8a7-4db2eaf42e1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:03.922888 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e069972c-cb18-4497-b8a7-4db2eaf42e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:03.923830 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e069972c-cb18-4497-b8a7-4db2eaf42e1f admin admin] PATH_INFO: `/v3/projects/168bfacc9fb549129401b8ed7fd59b87` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:03.927134 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e069972c-cb18-4497-b8a7-4db2eaf42e1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=168bfacc9fb549129401b8ed7fd59b87)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:03.930573 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e069972c-cb18-4497-b8a7-4db2eaf42e1f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:04.005715 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e069972c-cb18-4497-b8a7-4db2eaf42e1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '168bfacc9fb549129401b8ed7fd59b87', 'request_id': 'req-e069972c-cb18-4497-b8a7-4db2eaf42e1f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:04.018980 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e069972c-cb18-4497-b8a7-4db2eaf42e1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '168bfacc9fb549129401b8ed7fd59b87', 'request_id': 'req-e069972c-cb18-4497-b8a7-4db2eaf42e1f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:04.028758 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e069972c-cb18-4497-b8a7-4db2eaf42e1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '168bfacc9fb549129401b8ed7fd59b87', 'request_id': 'req-e069972c-cb18-4497-b8a7-4db2eaf42e1f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:04.052901 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e069972c-cb18-4497-b8a7-4db2eaf42e1f admin admin] The token cache is being invalidate because project 168bfacc9fb549129401b8ed7fd59b87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:41:04.053252 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e069972c-cb18-4497-b8a7-4db2eaf42e1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:04.053925 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e069972c-cb18-4497-b8a7-4db2eaf42e1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:04.056367 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1268/2575] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:41:03 2022] DELETE /identity/v3/projects/168bfacc9fb549129401b8ed7fd59b87 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:41:04.061273 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ec2ac8-8332-4168-829b-c51eed6ca39b None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:04.061967 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ec2ac8-8332-4168-829b-c51eed6ca39b None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:04.061967 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ec2ac8-8332-4168-829b-c51eed6ca39b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:04.099719 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-c3ec2ac8-8332-4168-829b-c51eed6ca39b None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:41:04.112589 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c3ec2ac8-8332-4168-829b-c51eed6ca39b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:04.286918 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1308/2576] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:41:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:41:04.293124 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9aec9-c8ce-47be-b751-f9135de6f5f6 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:04.293430 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9aec9-c8ce-47be-b751-f9135de6f5f6 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:04.293786 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9aec9-c8ce-47be-b751-f9135de6f5f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:04.331027 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f5e9aec9-c8ce-47be-b751-f9135de6f5f6 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:41:04.332817 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f5e9aec9-c8ce-47be-b751-f9135de6f5f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:04.360898 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1269/2577] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:41:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:41:04.363898 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ec2ac8-8332-4168-829b-c51eed6ca39b None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:04.421275 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ec2ac8-8332-4168-829b-c51eed6ca39b None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:04.435024 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeb2f29-da28-4c68-9270-f3bc25f9111e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:04.435731 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb2f29-da28-4c68-9270-f3bc25f9111e admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:04.436028 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb2f29-da28-4c68-9270-f3bc25f9111e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:04.436315 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb2f29-da28-4c68-9270-f3bc25f9111e admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:04.437132 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeb2f29-da28-4c68-9270-f3bc25f9111e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:41:04.446400 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeb2f29-da28-4c68-9270-f3bc25f9111e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:04.448074 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeb2f29-da28-4c68-9270-f3bc25f9111e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:04.465689 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1309/2578] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:41:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:41:04.468111 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e9aec9-c8ce-47be-b751-f9135de6f5f6 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:04.487066 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e9aec9-c8ce-47be-b751-f9135de6f5f6 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:04.507059 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481b62ef-1516-4040-98fc-c321dc6125f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:04.507968 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b62ef-1516-4040-98fc-c321dc6125f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:04.508606 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b62ef-1516-4040-98fc-c321dc6125f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:04.508937 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b62ef-1516-4040-98fc-c321dc6125f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:04.517981 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481b62ef-1516-4040-98fc-c321dc6125f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:04.517981 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481b62ef-1516-4040-98fc-c321dc6125f2 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:04.593518 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1270/2579] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:41:04 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:41:04.599082 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeb2f29-da28-4c68-9270-f3bc25f9111e admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:04.623774 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeb2f29-da28-4c68-9270-f3bc25f9111e admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:04.770985 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e001715a-f84c-44bf-985c-979f7cff9c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:04.772184 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001715a-f84c-44bf-985c-979f7cff9c68 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:04.772957 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001715a-f84c-44bf-985c-979f7cff9c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:04.773457 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001715a-f84c-44bf-985c-979f7cff9c68 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:04.774841 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e001715a-f84c-44bf-985c-979f7cff9c68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:04.778741 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e001715a-f84c-44bf-985c-979f7cff9c68 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:04.833467 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1310/2580] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:41:04 2022] POST /identity/v3/users => generated 293 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:41:04.846553 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481b62ef-1516-4040-98fc-c321dc6125f2 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:04.872717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481b62ef-1516-4040-98fc-c321dc6125f2 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:04.882700 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e6b1dd-6de0-412c-abd4-a8418238c6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:04.884490 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6b1dd-6de0-412c-abd4-a8418238c6f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:04.885223 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6b1dd-6de0-412c-abd4-a8418238c6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:04.885933 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6b1dd-6de0-412c-abd4-a8418238c6f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:04.887051 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e6b1dd-6de0-412c-abd4-a8418238c6f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:04.890058 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e6b1dd-6de0-412c-abd4-a8418238c6f3 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:04.913889 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1271/2581] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:41:04 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:41:04.920223 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e001715a-f84c-44bf-985c-979f7cff9c68 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:04.942354 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e001715a-f84c-44bf-985c-979f7cff9c68 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:04.955579 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747fc8b4-2672-4b18-b11c-e774053f0824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:04.956954 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747fc8b4-2672-4b18-b11c-e774053f0824 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:04.957761 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747fc8b4-2672-4b18-b11c-e774053f0824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:04.958601 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747fc8b4-2672-4b18-b11c-e774053f0824 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:04.960667 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747fc8b4-2672-4b18-b11c-e774053f0824 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:04.963442 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747fc8b4-2672-4b18-b11c-e774053f0824 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:04.993597 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1311/2582] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:41:04 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:41:04.999259 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e6b1dd-6de0-412c-abd4-a8418238c6f3 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:05.013747 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e6b1dd-6de0-412c-abd4-a8418238c6f3 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:05.024456 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde80e61-bb57-43c1-976c-7f177fc09514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:05.026455 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde80e61-bb57-43c1-976c-7f177fc09514 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:05.027139 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde80e61-bb57-43c1-976c-7f177fc09514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:05.027820 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde80e61-bb57-43c1-976c-7f177fc09514 admin admin] PATH_INFO: `/v3/projects/cc514ead89bf4578942144908727b1c7/users/cf44a58c69d644fa84fce2b1a98c0812/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:05.064531 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde80e61-bb57-43c1-976c-7f177fc09514 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc514ead89bf4578942144908727b1c7, user_id=cf44a58c69d644fa84fce2b1a98c0812, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:05.072510 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde80e61-bb57-43c1-976c-7f177fc09514 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:05.085372 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:41:05.106932 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1272/2583] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:41:04 2022] PUT /identity/v3/projects/cc514ead89bf4578942144908727b1c7/users/cf44a58c69d644fa84fce2b1a98c0812/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:41:05.126784 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553119d3-7d74-43a2-83e4-ec9475d1ae42 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:05.128644 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553119d3-7d74-43a2-83e4-ec9475d1ae42 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:05.128644 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553119d3-7d74-43a2-83e4-ec9475d1ae42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:05.188881 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-553119d3-7d74-43a2-83e4-ec9475d1ae42 None None] MFA Rules not processed for user `cf44a58c69d644fa84fce2b1a98c0812`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:41:05.191552 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-553119d3-7d74-43a2-83e4-ec9475d1ae42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:05.290576 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:41:05.332962 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1312/2584] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:41:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:41:05.349167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde80e61-bb57-43c1-976c-7f177fc09514 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:05.352188 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-bde80e61-bb57-43c1-976c-7f177fc09514 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:05.488038 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde80e61-bb57-43c1-976c-7f177fc09514 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:05.491940 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ebcc6-71d7-4b3a-accc-27baf1dee58b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:05.492760 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ebcc6-71d7-4b3a-accc-27baf1dee58b service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:05.493053 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ebcc6-71d7-4b3a-accc-27baf1dee58b service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:05.493417 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ebcc6-71d7-4b3a-accc-27baf1dee58b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:05.504387 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2ebcc6-71d7-4b3a-accc-27baf1dee58b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:05.506297 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2ebcc6-71d7-4b3a-accc-27baf1dee58b service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:05.519274 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1273/2585] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:41:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:41:05.557684 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553119d3-7d74-43a2-83e4-ec9475d1ae42 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:05.825293 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553119d3-7d74-43a2-83e4-ec9475d1ae42 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:05.831980 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87d66fa-553d-4976-9d86-d4b4bc0513ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:05.832897 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d66fa-553d-4976-9d86-d4b4bc0513ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:05.833183 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d66fa-553d-4976-9d86-d4b4bc0513ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:05.833593 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d66fa-553d-4976-9d86-d4b4bc0513ef admin admin] PATH_INFO: `/v3/users/cf44a58c69d644fa84fce2b1a98c0812` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:05.835030 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87d66fa-553d-4976-9d86-d4b4bc0513ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf44a58c69d644fa84fce2b1a98c0812)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:05.837130 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87d66fa-553d-4976-9d86-d4b4bc0513ef admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:05.925866 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-a87d66fa-553d-4976-9d86-d4b4bc0513ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf44a58c69d644fa84fce2b1a98c0812'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:05.942554 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-a87d66fa-553d-4976-9d86-d4b4bc0513ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf44a58c69d644fa84fce2b1a98c0812'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:05.944329 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-a87d66fa-553d-4976-9d86-d4b4bc0513ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf44a58c69d644fa84fce2b1a98c0812'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:05.946491 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-a87d66fa-553d-4976-9d86-d4b4bc0513ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf44a58c69d644fa84fce2b1a98c0812'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:05.957786 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-a87d66fa-553d-4976-9d86-d4b4bc0513ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf44a58c69d644fa84fce2b1a98c0812'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:05.999489 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1313/2586] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:41:05 2022] DELETE /identity/v3/users/cf44a58c69d644fa84fce2b1a98c0812 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:41:06.007259 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77fc0f-dfa9-4e5f-8b1b-d0400b25ad53 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:06.008026 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77fc0f-dfa9-4e5f-8b1b-d0400b25ad53 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:06.008268 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77fc0f-dfa9-4e5f-8b1b-d0400b25ad53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:06.031531 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-9d77fc0f-dfa9-4e5f-8b1b-d0400b25ad53 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:41:06.032839 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9d77fc0f-dfa9-4e5f-8b1b-d0400b25ad53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:06.128822 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1274/2587] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:41:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:41:06.139353 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87d66fa-553d-4976-9d86-d4b4bc0513ef admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:06.141007 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a87d66fa-553d-4976-9d86-d4b4bc0513ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:06.211000 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87d66fa-553d-4976-9d86-d4b4bc0513ef admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:06.215444 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beefa184-ae1e-4521-b40f-47345035a5aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:06.216315 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beefa184-ae1e-4521-b40f-47345035a5aa service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:06.216655 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beefa184-ae1e-4521-b40f-47345035a5aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:06.217028 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beefa184-ae1e-4521-b40f-47345035a5aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:06.228417 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beefa184-ae1e-4521-b40f-47345035a5aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:06.229981 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beefa184-ae1e-4521-b40f-47345035a5aa service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:06.242596 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1314/2588] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:41:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:41:06.686817 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d77fc0f-dfa9-4e5f-8b1b-d0400b25ad53 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:06.689407 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9d77fc0f-dfa9-4e5f-8b1b-d0400b25ad53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:06.728876 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d77fc0f-dfa9-4e5f-8b1b-d0400b25ad53 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:06.740423 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826203ec-2f2a-4328-80fb-71f5e0342469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:06.742592 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826203ec-2f2a-4328-80fb-71f5e0342469 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:06.743224 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826203ec-2f2a-4328-80fb-71f5e0342469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:06.743662 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826203ec-2f2a-4328-80fb-71f5e0342469 admin admin] PATH_INFO: `/v3/projects/cc514ead89bf4578942144908727b1c7` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:06.746632 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826203ec-2f2a-4328-80fb-71f5e0342469 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc514ead89bf4578942144908727b1c7)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:06.750853 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826203ec-2f2a-4328-80fb-71f5e0342469 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:06.822459 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-826203ec-2f2a-4328-80fb-71f5e0342469 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc514ead89bf4578942144908727b1c7', 'request_id': 'req-826203ec-2f2a-4328-80fb-71f5e0342469'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:06.833917 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-826203ec-2f2a-4328-80fb-71f5e0342469 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc514ead89bf4578942144908727b1c7', 'request_id': 'req-826203ec-2f2a-4328-80fb-71f5e0342469'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:06.850662 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-826203ec-2f2a-4328-80fb-71f5e0342469 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc514ead89bf4578942144908727b1c7', 'request_id': 'req-826203ec-2f2a-4328-80fb-71f5e0342469'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:06.866350 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-826203ec-2f2a-4328-80fb-71f5e0342469 admin admin] The token cache is being invalidate because project cc514ead89bf4578942144908727b1c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:41:06.866749 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-826203ec-2f2a-4328-80fb-71f5e0342469 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:06.867566 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-826203ec-2f2a-4328-80fb-71f5e0342469 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:06.869664 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1275/2589] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:41:06 2022] DELETE /identity/v3/projects/cc514ead89bf4578942144908727b1c7 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:41:06.876285 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba80a4f-3acf-4c66-bf7e-bbf69373be47 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:06.878421 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba80a4f-3acf-4c66-bf7e-bbf69373be47 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:06.879014 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba80a4f-3acf-4c66-bf7e-bbf69373be47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:06.904254 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-1ba80a4f-3acf-4c66-bf7e-bbf69373be47 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:41:06.905858 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-1ba80a4f-3acf-4c66-bf7e-bbf69373be47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:06.918028 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:41:07.014792 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1315/2590] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:41:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:41:07.017443 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90c438-c8fa-4651-bff7-815540c38032 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:07.018828 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90c438-c8fa-4651-bff7-815540c38032 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:07.019281 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90c438-c8fa-4651-bff7-815540c38032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:07.052627 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-5a90c438-c8fa-4651-bff7-815540c38032 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:41:07.054104 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5a90c438-c8fa-4651-bff7-815540c38032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:07.063167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1276/2591] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:41:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:41:07.069897 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba80a4f-3acf-4c66-bf7e-bbf69373be47 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:07.083802 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba80a4f-3acf-4c66-bf7e-bbf69373be47 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:07.094051 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f6422e-803b-4d77-a8a0-95e19e3a48fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:07.095283 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f6422e-803b-4d77-a8a0-95e19e3a48fa admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:07.096818 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f6422e-803b-4d77-a8a0-95e19e3a48fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:07.097916 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f6422e-803b-4d77-a8a0-95e19e3a48fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:07.099467 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f6422e-803b-4d77-a8a0-95e19e3a48fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:41:07.100195 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f6422e-803b-4d77-a8a0-95e19e3a48fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:07.103354 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f6422e-803b-4d77-a8a0-95e19e3a48fa admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:07.129768 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1316/2592] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:41:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:41:07.137013 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90c438-c8fa-4651-bff7-815540c38032 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:07.153914 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90c438-c8fa-4651-bff7-815540c38032 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:07.162212 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf784c3-9921-4a5e-a0bc-0e2a6cdaf437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:07.163239 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf784c3-9921-4a5e-a0bc-0e2a6cdaf437 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:07.163690 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf784c3-9921-4a5e-a0bc-0e2a6cdaf437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:07.164224 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf784c3-9921-4a5e-a0bc-0e2a6cdaf437 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:07.165412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf784c3-9921-4a5e-a0bc-0e2a6cdaf437 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:07.168991 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf784c3-9921-4a5e-a0bc-0e2a6cdaf437 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:07.200817 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:41:07.227434 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1277/2593] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:41:07 2022] POST /identity/v3/projects => generated 363 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:41:07.236371 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f6422e-803b-4d77-a8a0-95e19e3a48fa admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:07.255588 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f6422e-803b-4d77-a8a0-95e19e3a48fa admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:07.383865 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcd4415-c1e6-4b77-8fc8-1c825831a37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:07.385073 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd4415-c1e6-4b77-8fc8-1c825831a37e admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:07.385421 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd4415-c1e6-4b77-8fc8-1c825831a37e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:07.386119 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd4415-c1e6-4b77-8fc8-1c825831a37e admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:07.387319 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcd4415-c1e6-4b77-8fc8-1c825831a37e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:07.389687 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcd4415-c1e6-4b77-8fc8-1c825831a37e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:07.426000 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1317/2594] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:41:07 2022] POST /identity/v3/users => generated 282 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:41:07.432148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf784c3-9921-4a5e-a0bc-0e2a6cdaf437 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:07.447589 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf784c3-9921-4a5e-a0bc-0e2a6cdaf437 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:07.453120 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4139fa3-c564-4287-b56b-f6134cd55d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:07.454248 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4139fa3-c564-4287-b56b-f6134cd55d10 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:07.454547 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4139fa3-c564-4287-b56b-f6134cd55d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:07.454953 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4139fa3-c564-4287-b56b-f6134cd55d10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:07.456109 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4139fa3-c564-4287-b56b-f6134cd55d10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:07.457890 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4139fa3-c564-4287-b56b-f6134cd55d10 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:07.470980 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1278/2595] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:41:07 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:41:07.475592 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcd4415-c1e6-4b77-8fc8-1c825831a37e admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:07.487902 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcd4415-c1e6-4b77-8fc8-1c825831a37e admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:07.500193 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a448c99-8dbc-406b-a3f3-afee1fc734e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:07.500921 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a448c99-8dbc-406b-a3f3-afee1fc734e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:07.501206 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a448c99-8dbc-406b-a3f3-afee1fc734e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:07.501487 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a448c99-8dbc-406b-a3f3-afee1fc734e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:07.502643 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a448c99-8dbc-406b-a3f3-afee1fc734e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:07.504330 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a448c99-8dbc-406b-a3f3-afee1fc734e6 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:07.519835 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1318/2596] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:41:07 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:41:07.526278 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4139fa3-c564-4287-b56b-f6134cd55d10 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:07.538555 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4139fa3-c564-4287-b56b-f6134cd55d10 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:07.544775 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ef6c4b-6387-4998-baf0-ee446e284ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:07.545753 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef6c4b-6387-4998-baf0-ee446e284ab6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:07.546050 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef6c4b-6387-4998-baf0-ee446e284ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:07.546341 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef6c4b-6387-4998-baf0-ee446e284ab6 admin admin] PATH_INFO: `/v3/projects/63425ce0072941c9a3d59cd111fe691d/users/757b04583c444e2aa302d1078908a312/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:07.565695 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ef6c4b-6387-4998-baf0-ee446e284ab6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63425ce0072941c9a3d59cd111fe691d, user_id=757b04583c444e2aa302d1078908a312, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:07.568555 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ef6c4b-6387-4998-baf0-ee446e284ab6 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:07.586544 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1279/2597] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:41:07 2022] PUT /identity/v3/projects/63425ce0072941c9a3d59cd111fe691d/users/757b04583c444e2aa302d1078908a312/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:41:07.597966 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b6e1a3-283a-4e8d-a21c-e047176b5907 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:07.598267 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b6e1a3-283a-4e8d-a21c-e047176b5907 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:07.598524 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b6e1a3-283a-4e8d-a21c-e047176b5907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:07.631441 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-34b6e1a3-283a-4e8d-a21c-e047176b5907 None None] MFA Rules not processed for user `757b04583c444e2aa302d1078908a312`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:41:07.632703 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-34b6e1a3-283a-4e8d-a21c-e047176b5907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:07.715203 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1319/2598] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:41:07 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:41:07.729157 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ef6c4b-6387-4998-baf0-ee446e284ab6 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:07.731673 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b9ef6c4b-6387-4998-baf0-ee446e284ab6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:07.806080 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ef6c4b-6387-4998-baf0-ee446e284ab6 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:07.810072 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ab8d4-c925-4d50-a855-dd447736a40f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:07.810738 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ab8d4-c925-4d50-a855-dd447736a40f service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:07.811043 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ab8d4-c925-4d50-a855-dd447736a40f service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:07.811352 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ab8d4-c925-4d50-a855-dd447736a40f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:07.823000 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852ab8d4-c925-4d50-a855-dd447736a40f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:07.824527 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852ab8d4-c925-4d50-a855-dd447736a40f service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:07.836118 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1280/2599] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:41:07 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:41:13.780657 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b6e1a3-283a-4e8d-a21c-e047176b5907 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:13.783159 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-34b6e1a3-283a-4e8d-a21c-e047176b5907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:13.813143 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:41:14.090238 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b6e1a3-283a-4e8d-a21c-e047176b5907 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:14.099534 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be26acaf-8733-4ca4-9995-c6fbcbbfa5ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:14.100671 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be26acaf-8733-4ca4-9995-c6fbcbbfa5ce service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:14.101176 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be26acaf-8733-4ca4-9995-c6fbcbbfa5ce service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:14.101669 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be26acaf-8733-4ca4-9995-c6fbcbbfa5ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:14.105213 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-be26acaf-8733-4ca4-9995-c6fbcbbfa5ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:14.122814 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be26acaf-8733-4ca4-9995-c6fbcbbfa5ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:14.125707 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be26acaf-8733-4ca4-9995-c6fbcbbfa5ce service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:14.146926 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1320/2600] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:41:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:41:17.831153 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:INSERT=2,keystone:SELECT=43 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:41:23.452413 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ab8d4-c925-4d50-a855-dd447736a40f service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:23.454888 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-852ab8d4-c925-4d50-a855-dd447736a40f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:23.525737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ab8d4-c925-4d50-a855-dd447736a40f service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:23.529962 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7480558-58ff-4d2d-a230-d17091be2aa5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:23.530679 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7480558-58ff-4d2d-a230-d17091be2aa5 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:23.530971 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7480558-58ff-4d2d-a230-d17091be2aa5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:23.531250 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7480558-58ff-4d2d-a230-d17091be2aa5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:23.533599 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a7480558-58ff-4d2d-a230-d17091be2aa5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:23.641840 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7480558-58ff-4d2d-a230-d17091be2aa5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:23.644575 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7480558-58ff-4d2d-a230-d17091be2aa5 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:23.663877 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1281/2601] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:41:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:41:24.138781 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=39 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:41:26.011146 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be26acaf-8733-4ca4-9995-c6fbcbbfa5ce service nova] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:26.034097 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be26acaf-8733-4ca4-9995-c6fbcbbfa5ce service nova] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:26.046522 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e22de9-de2d-497c-8504-8a69dc30d02e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:26.047828 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e22de9-de2d-497c-8504-8a69dc30d02e service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:26.048329 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e22de9-de2d-497c-8504-8a69dc30d02e service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:26.048814 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e22de9-de2d-497c-8504-8a69dc30d02e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:26.057251 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a3e22de9-de2d-497c-8504-8a69dc30d02e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:26.088807 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e22de9-de2d-497c-8504-8a69dc30d02e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:26.091978 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e22de9-de2d-497c-8504-8a69dc30d02e service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:26.125745 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1321/2602] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:41:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:41:29.184511 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8bb9b-0c57-423c-8555-7723cbbcaf7c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:29.185252 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8bb9b-0c57-423c-8555-7723cbbcaf7c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:29.185378 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8bb9b-0c57-423c-8555-7723cbbcaf7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:29.259306 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-14b8bb9b-0c57-423c-8555-7723cbbcaf7c None None] MFA Rules not processed for user `09c9fa7dccb8491c9ad3b43967345ea2`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:41:29.261055 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-14b8bb9b-0c57-423c-8555-7723cbbcaf7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:29.388460 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1282/2603] 10.222.0.87 () {58 vars in 967 bytes} [Tue Jun 21 18:41:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:41:29.433147 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e22de9-de2d-497c-8504-8a69dc30d02e service nova] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:29.434811 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a3e22de9-de2d-497c-8504-8a69dc30d02e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:29.621869 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e22de9-de2d-497c-8504-8a69dc30d02e service nova] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:29.634529 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7e9132-94b5-4228-a9c5-7156ca4c51b4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '515b3571045e4fe29565307c8ecc10b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:29.635700 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e9132-94b5-4228-a9c5-7156ca4c51b4 service swift] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:29.635820 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e9132-94b5-4228-a9c5-7156ca4c51b4 service swift] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:29.636243 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e9132-94b5-4228-a9c5-7156ca4c51b4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:29.642805 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b8bb9b-0c57-423c-8555-7723cbbcaf7c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:29.644867 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-14b8bb9b-0c57-423c-8555-7723cbbcaf7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:29.653544 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7e9132-94b5-4228-a9c5-7156ca4c51b4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:29.655867 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7e9132-94b5-4228-a9c5-7156ca4c51b4 service swift] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:29.667513 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1322/2604] 10.222.0.87 () {60 vars in 1384 bytes} [Tue Jun 21 18:41:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 18:41:29.679875 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b8bb9b-0c57-423c-8555-7723cbbcaf7c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:29.687872 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4ebbe9-91c3-4994-9f92-e0eab0fd1436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:29.689971 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ebbe9-91c3-4994-9f92-e0eab0fd1436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:29.691234 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ebbe9-91c3-4994-9f92-e0eab0fd1436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:29.692242 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ebbe9-91c3-4994-9f92-e0eab0fd1436 admin admin] PATH_INFO: `/v3/users/144c9c31ed9b469aa200dcb16c89ab4e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:29.695195 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4ebbe9-91c3-4994-9f92-e0eab0fd1436 admin admin] RBAC: Authorizing `identity:delete_user(user_id=144c9c31ed9b469aa200dcb16c89ab4e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:29.699093 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4ebbe9-91c3-4994-9f92-e0eab0fd1436 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:29.796136 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2f4ebbe9-91c3-4994-9f92-e0eab0fd1436 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '144c9c31ed9b469aa200dcb16c89ab4e'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:29.799660 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:41:29.819729 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2f4ebbe9-91c3-4994-9f92-e0eab0fd1436 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '144c9c31ed9b469aa200dcb16c89ab4e'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:29.822270 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2f4ebbe9-91c3-4994-9f92-e0eab0fd1436 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '144c9c31ed9b469aa200dcb16c89ab4e'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:29.822450 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2f4ebbe9-91c3-4994-9f92-e0eab0fd1436 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '144c9c31ed9b469aa200dcb16c89ab4e'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:29.848900 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2f4ebbe9-91c3-4994-9f92-e0eab0fd1436 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '144c9c31ed9b469aa200dcb16c89ab4e'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:29.876183 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1283/2605] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:41:29 2022] DELETE /identity/v3/users/144c9c31ed9b469aa200dcb16c89ab4e => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:41:30.204820 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7e9132-94b5-4228-a9c5-7156ca4c51b4 service swift] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:30.226152 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7e9132-94b5-4228-a9c5-7156ca4c51b4 service swift] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:30.330697 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e50205a-5f21-4029-bab8-22023aab14dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:30.332615 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e50205a-5f21-4029-bab8-22023aab14dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:30.333112 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e50205a-5f21-4029-bab8-22023aab14dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:30.333603 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e50205a-5f21-4029-bab8-22023aab14dc admin admin] PATH_INFO: `/v3/projects/53e2bb9f793041c2aef6e222b76c3961` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:30.335973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e50205a-5f21-4029-bab8-22023aab14dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=53e2bb9f793041c2aef6e222b76c3961)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:30.340235 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e50205a-5f21-4029-bab8-22023aab14dc admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:30.436836 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-6e50205a-5f21-4029-bab8-22023aab14dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53e2bb9f793041c2aef6e222b76c3961', 'request_id': 'req-6e50205a-5f21-4029-bab8-22023aab14dc'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:30.447100 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-6e50205a-5f21-4029-bab8-22023aab14dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53e2bb9f793041c2aef6e222b76c3961', 'request_id': 'req-6e50205a-5f21-4029-bab8-22023aab14dc'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:30.461509 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-6e50205a-5f21-4029-bab8-22023aab14dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53e2bb9f793041c2aef6e222b76c3961', 'request_id': 'req-6e50205a-5f21-4029-bab8-22023aab14dc'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:30.483192 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-6e50205a-5f21-4029-bab8-22023aab14dc admin admin] The token cache is being invalidate because project 53e2bb9f793041c2aef6e222b76c3961 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:41:30.483694 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-6e50205a-5f21-4029-bab8-22023aab14dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:30.484638 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-6e50205a-5f21-4029-bab8-22023aab14dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:30.487934 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1323/2606] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:41:30 2022] DELETE /identity/v3/projects/53e2bb9f793041c2aef6e222b76c3961 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:41:30.496117 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f6f8e4-c756-4adf-8aa3-fe06ffe3d2bd None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:30.501776 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f6f8e4-c756-4adf-8aa3-fe06ffe3d2bd None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:30.502390 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f6f8e4-c756-4adf-8aa3-fe06ffe3d2bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:30.535714 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-30f6f8e4-c756-4adf-8aa3-fe06ffe3d2bd None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:41:30.537403 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-30f6f8e4-c756-4adf-8aa3-fe06ffe3d2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:30.636495 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1284/2607] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:41:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:41:30.648103 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a03e55-95c0-4156-b181-bb3df9e84ab7 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:30.648958 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a03e55-95c0-4156-b181-bb3df9e84ab7 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:30.649893 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a03e55-95c0-4156-b181-bb3df9e84ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:30.667605 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:41:30.677416 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-74a03e55-95c0-4156-b181-bb3df9e84ab7 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:41:30.680807 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-74a03e55-95c0-4156-b181-bb3df9e84ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:30.692362 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1324/2608] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:41:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:41:30.697687 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f6f8e4-c756-4adf-8aa3-fe06ffe3d2bd None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:30.711854 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f6f8e4-c756-4adf-8aa3-fe06ffe3d2bd None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:30.724379 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f798d0-d29d-4311-b4a4-63285701c299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:30.726263 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f798d0-d29d-4311-b4a4-63285701c299 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:30.727415 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f798d0-d29d-4311-b4a4-63285701c299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:30.727960 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f798d0-d29d-4311-b4a4-63285701c299 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:30.729022 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f798d0-d29d-4311-b4a4-63285701c299 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:41:30.729534 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f798d0-d29d-4311-b4a4-63285701c299 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:30.732127 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f798d0-d29d-4311-b4a4-63285701c299 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:30.756471 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1285/2609] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:41:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:41:30.761783 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a03e55-95c0-4156-b181-bb3df9e84ab7 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:30.778173 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a03e55-95c0-4156-b181-bb3df9e84ab7 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:30.791510 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3aac6d-447a-46b9-9683-9d3c01db1418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:30.792574 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3aac6d-447a-46b9-9683-9d3c01db1418 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:30.792979 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3aac6d-447a-46b9-9683-9d3c01db1418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:30.793351 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3aac6d-447a-46b9-9683-9d3c01db1418 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:30.794263 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3aac6d-447a-46b9-9683-9d3c01db1418 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:30.796665 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3aac6d-447a-46b9-9683-9d3c01db1418 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:30.845932 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1325/2610] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:41:30 2022] POST /identity/v3/projects => generated 379 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:41:30.849574 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f798d0-d29d-4311-b4a4-63285701c299 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:30.870841 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f798d0-d29d-4311-b4a4-63285701c299 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:30.950240 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41419d3-2367-488e-b943-0897dbab002e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:30.950888 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41419d3-2367-488e-b943-0897dbab002e admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:30.951301 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41419d3-2367-488e-b943-0897dbab002e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:30.951673 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41419d3-2367-488e-b943-0897dbab002e admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:30.952444 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41419d3-2367-488e-b943-0897dbab002e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:30.955292 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41419d3-2367-488e-b943-0897dbab002e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:30.982332 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1286/2611] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:41:30 2022] POST /identity/v3/users => generated 290 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:41:30.988206 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3aac6d-447a-46b9-9683-9d3c01db1418 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:31.000340 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3aac6d-447a-46b9-9683-9d3c01db1418 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:31.006320 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5630be63-a8cf-4f9e-bb37-e8197db90307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:31.007085 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630be63-a8cf-4f9e-bb37-e8197db90307 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:31.007407 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630be63-a8cf-4f9e-bb37-e8197db90307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:31.007723 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630be63-a8cf-4f9e-bb37-e8197db90307 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:31.008503 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5630be63-a8cf-4f9e-bb37-e8197db90307 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:31.010265 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5630be63-a8cf-4f9e-bb37-e8197db90307 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:31.026098 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1326/2612] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:41:30 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:41:31.036222 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41419d3-2367-488e-b943-0897dbab002e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:31.047721 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41419d3-2367-488e-b943-0897dbab002e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:31.053753 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07466b59-b975-46ee-8b7c-4de52af033ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:31.054539 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07466b59-b975-46ee-8b7c-4de52af033ea admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:31.055007 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07466b59-b975-46ee-8b7c-4de52af033ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:31.055340 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07466b59-b975-46ee-8b7c-4de52af033ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:31.056145 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07466b59-b975-46ee-8b7c-4de52af033ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:31.057981 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07466b59-b975-46ee-8b7c-4de52af033ea admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:31.076966 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1287/2613] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:41:31.078069 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5630be63-a8cf-4f9e-bb37-e8197db90307 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:31.090704 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5630be63-a8cf-4f9e-bb37-e8197db90307 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:31.095930 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d8f04-955f-4346-9afb-ac253da6e9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:31.097518 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d8f04-955f-4346-9afb-ac253da6e9c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:31.097857 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d8f04-955f-4346-9afb-ac253da6e9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:31.098202 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d8f04-955f-4346-9afb-ac253da6e9c6 admin admin] PATH_INFO: `/v3/projects/10e4801c1ef1487d90e1b71b88475355/users/f40b58a0f7294c32bfb18fa672ec9279/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:31.118721 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6d8f04-955f-4346-9afb-ac253da6e9c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10e4801c1ef1487d90e1b71b88475355, user_id=f40b58a0f7294c32bfb18fa672ec9279, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:31.121715 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6d8f04-955f-4346-9afb-ac253da6e9c6 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:31.139190 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1327/2614] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:41:31 2022] PUT /identity/v3/projects/10e4801c1ef1487d90e1b71b88475355/users/f40b58a0f7294c32bfb18fa672ec9279/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:41:31.152019 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605345e5-3c27-4084-97ce-b4624acdb28b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:31.152433 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605345e5-3c27-4084-97ce-b4624acdb28b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:31.152836 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605345e5-3c27-4084-97ce-b4624acdb28b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:31.192869 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-605345e5-3c27-4084-97ce-b4624acdb28b None None] MFA Rules not processed for user `f40b58a0f7294c32bfb18fa672ec9279`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:41:31.194285 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-605345e5-3c27-4084-97ce-b4624acdb28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:31.224377 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:41:31.299541 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1288/2615] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:41:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:41:31.304872 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d8f04-955f-4346-9afb-ac253da6e9c6 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:31.385277 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d8f04-955f-4346-9afb-ac253da6e9c6 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:31.391005 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630317ba-93a2-4b53-83a4-1ba6a8b3663c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:31.391702 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630317ba-93a2-4b53-83a4-1ba6a8b3663c admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:31.392057 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630317ba-93a2-4b53-83a4-1ba6a8b3663c admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:31.392495 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630317ba-93a2-4b53-83a4-1ba6a8b3663c admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:31.393898 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630317ba-93a2-4b53-83a4-1ba6a8b3663c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:31.396772 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630317ba-93a2-4b53-83a4-1ba6a8b3663c admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:31.453527 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1328/2616] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:41:31 2022] POST /identity/v3/projects => generated 379 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:41:31.455714 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605345e5-3c27-4084-97ce-b4624acdb28b None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:31.477431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605345e5-3c27-4084-97ce-b4624acdb28b None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:31.562462 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90538dd-495b-4048-9fd6-aa5f27d8d3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:31.563559 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90538dd-495b-4048-9fd6-aa5f27d8d3c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:31.563559 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90538dd-495b-4048-9fd6-aa5f27d8d3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:31.563909 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90538dd-495b-4048-9fd6-aa5f27d8d3c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:31.564574 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90538dd-495b-4048-9fd6-aa5f27d8d3c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:31.566832 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90538dd-495b-4048-9fd6-aa5f27d8d3c3 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:31.597364 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1289/2617] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:41:31 2022] POST /identity/v3/users => generated 296 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:41:31.603423 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630317ba-93a2-4b53-83a4-1ba6a8b3663c admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:31.623831 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630317ba-93a2-4b53-83a4-1ba6a8b3663c admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:31.629435 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439f73b-1782-44fa-8b80-eb7d1b39ebcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:31.630201 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439f73b-1782-44fa-8b80-eb7d1b39ebcd admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:31.630576 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439f73b-1782-44fa-8b80-eb7d1b39ebcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:31.630992 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439f73b-1782-44fa-8b80-eb7d1b39ebcd admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:31.632206 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4439f73b-1782-44fa-8b80-eb7d1b39ebcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:31.634149 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4439f73b-1782-44fa-8b80-eb7d1b39ebcd admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:31.649330 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1329/2618] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:41:31.656548 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90538dd-495b-4048-9fd6-aa5f27d8d3c3 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:31.679398 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90538dd-495b-4048-9fd6-aa5f27d8d3c3 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:31.687837 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280bbdd7-6b23-4e01-8ba2-b70191d3c399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:31.689499 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280bbdd7-6b23-4e01-8ba2-b70191d3c399 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:31.690137 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280bbdd7-6b23-4e01-8ba2-b70191d3c399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:31.690643 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280bbdd7-6b23-4e01-8ba2-b70191d3c399 admin admin] PATH_INFO: `/v3/projects/a3835eeee87b422bb3c32248602f145c/users/64a42a28a66a405d9c4806a55cc8386f/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:31.721328 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280bbdd7-6b23-4e01-8ba2-b70191d3c399 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3835eeee87b422bb3c32248602f145c, user_id=64a42a28a66a405d9c4806a55cc8386f, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:31.726823 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280bbdd7-6b23-4e01-8ba2-b70191d3c399 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:31.759286 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1290/2619] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:41:31 2022] PUT /identity/v3/projects/a3835eeee87b422bb3c32248602f145c/users/64a42a28a66a405d9c4806a55cc8386f/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:41:31.764842 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439f73b-1782-44fa-8b80-eb7d1b39ebcd admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:31.783117 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439f73b-1782-44fa-8b80-eb7d1b39ebcd admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:31.883876 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fed459-a0aa-4dac-8ab2-11873f6390ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:31.885042 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fed459-a0aa-4dac-8ab2-11873f6390ff admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:31.885476 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fed459-a0aa-4dac-8ab2-11873f6390ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:31.885747 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fed459-a0aa-4dac-8ab2-11873f6390ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:31.886645 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fed459-a0aa-4dac-8ab2-11873f6390ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:31.889601 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fed459-a0aa-4dac-8ab2-11873f6390ff admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:31.902624 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1330/2620] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:41:31.907532 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280bbdd7-6b23-4e01-8ba2-b70191d3c399 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:31.922318 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280bbdd7-6b23-4e01-8ba2-b70191d3c399 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:31.928077 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6fed57-b6bf-4dc3-912a-9cfbe7771422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:31.928996 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6fed57-b6bf-4dc3-912a-9cfbe7771422 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:31.929274 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6fed57-b6bf-4dc3-912a-9cfbe7771422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:31.929548 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6fed57-b6bf-4dc3-912a-9cfbe7771422 admin admin] PATH_INFO: `/v3/projects/a3835eeee87b422bb3c32248602f145c/users/64a42a28a66a405d9c4806a55cc8386f/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:31.931739 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6fed57-b6bf-4dc3-912a-9cfbe7771422 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3835eeee87b422bb3c32248602f145c, user_id=64a42a28a66a405d9c4806a55cc8386f, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:31.934426 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6fed57-b6bf-4dc3-912a-9cfbe7771422 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:31.947751 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1291/2621] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:41:31 2022] PUT /identity/v3/projects/a3835eeee87b422bb3c32248602f145c/users/64a42a28a66a405d9c4806a55cc8386f/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:41:31.962898 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bda4f0-298c-4faa-95e8-6fd792fe3dbc None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:31.962898 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bda4f0-298c-4faa-95e8-6fd792fe3dbc None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:31.963695 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bda4f0-298c-4faa-95e8-6fd792fe3dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:32.004137 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:41:32.011711 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-10bda4f0-298c-4faa-95e8-6fd792fe3dbc None None] MFA Rules not processed for user `64a42a28a66a405d9c4806a55cc8386f`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:41:32.013867 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-10bda4f0-298c-4faa-95e8-6fd792fe3dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:32.141054 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1331/2622] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:41:31 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:41:32.154950 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6fed57-b6bf-4dc3-912a-9cfbe7771422 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:32.157711 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9d6fed57-b6bf-4dc3-912a-9cfbe7771422 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:32.247616 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6fed57-b6bf-4dc3-912a-9cfbe7771422 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:32.251549 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f1c2df-6957-4752-9657-c5cc736ec9b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:32.252359 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f1c2df-6957-4752-9657-c5cc736ec9b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:32.252778 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f1c2df-6957-4752-9657-c5cc736ec9b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:32.253257 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f1c2df-6957-4752-9657-c5cc736ec9b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:32.265048 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f1c2df-6957-4752-9657-c5cc736ec9b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:32.266477 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f1c2df-6957-4752-9657-c5cc736ec9b4 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:32.281752 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1292/2623] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:41:32 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:41:32.759727 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bda4f0-298c-4faa-95e8-6fd792fe3dbc None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:32.777826 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bda4f0-298c-4faa-95e8-6fd792fe3dbc None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:32.786289 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce7419-092b-4ac2-bbb9-3c592f30b776 tempest-VolumeQuotaClassesTest-671961166 tempest-VolumeQuotaClassesTest-671961166-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42a28a66a405d9c4806a55cc8386f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3835eeee87b422bb3c32248602f145c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:32.787217 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce7419-092b-4ac2-bbb9-3c592f30b776 tempest-VolumeQuotaClassesTest-671961166 tempest-VolumeQuotaClassesTest-671961166-project-admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:32.787527 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce7419-092b-4ac2-bbb9-3c592f30b776 tempest-VolumeQuotaClassesTest-671961166 tempest-VolumeQuotaClassesTest-671961166-project-admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:32.787948 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce7419-092b-4ac2-bbb9-3c592f30b776 tempest-VolumeQuotaClassesTest-671961166 tempest-VolumeQuotaClassesTest-671961166-project-admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:32.788879 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ce7419-092b-4ac2-bbb9-3c592f30b776 tempest-VolumeQuotaClassesTest-671961166 tempest-VolumeQuotaClassesTest-671961166-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:41:32.789438 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ce7419-092b-4ac2-bbb9-3c592f30b776 tempest-VolumeQuotaClassesTest-671961166 tempest-VolumeQuotaClassesTest-671961166-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:32.792358 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ce7419-092b-4ac2-bbb9-3c592f30b776 tempest-VolumeQuotaClassesTest-671961166 tempest-VolumeQuotaClassesTest-671961166-project-admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:32.816094 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1332/2624] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:41:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:41:32.822539 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f1c2df-6957-4752-9657-c5cc736ec9b4 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:32.836939 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f1c2df-6957-4752-9657-c5cc736ec9b4 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:32.844004 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf9a327-41fa-4ffb-968f-247576ee1ab6 tempest-VolumeQuotaClassesTest-671961166 tempest-VolumeQuotaClassesTest-671961166-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42a28a66a405d9c4806a55cc8386f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3835eeee87b422bb3c32248602f145c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:32.844745 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9a327-41fa-4ffb-968f-247576ee1ab6 tempest-VolumeQuotaClassesTest-671961166 tempest-VolumeQuotaClassesTest-671961166-project-admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:32.845065 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9a327-41fa-4ffb-968f-247576ee1ab6 tempest-VolumeQuotaClassesTest-671961166 tempest-VolumeQuotaClassesTest-671961166-project-admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:32.845478 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9a327-41fa-4ffb-968f-247576ee1ab6 tempest-VolumeQuotaClassesTest-671961166 tempest-VolumeQuotaClassesTest-671961166-project-admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:32.846566 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf9a327-41fa-4ffb-968f-247576ee1ab6 tempest-VolumeQuotaClassesTest-671961166 tempest-VolumeQuotaClassesTest-671961166-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:32.848343 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf9a327-41fa-4ffb-968f-247576ee1ab6 tempest-VolumeQuotaClassesTest-671961166 tempest-VolumeQuotaClassesTest-671961166-project-admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:32.869448 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:41:32.887286 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1293/2625] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:41:32 2022] POST /identity/v3/projects => generated 354 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:41:32.892103 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce7419-092b-4ac2-bbb9-3c592f30b776 tempest-VolumeQuotaClassesTest-671961166 tempest-VolumeQuotaClassesTest-671961166-project-admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:32.906588 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce7419-092b-4ac2-bbb9-3c592f30b776 tempest-VolumeQuotaClassesTest-671961166 tempest-VolumeQuotaClassesTest-671961166-project-admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:33.010699 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac3d3c-9951-4bc9-b7b8-eba1747743c7 tempest-VolumeQuotaClassesTest-671961166 tempest-VolumeQuotaClassesTest-671961166-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42a28a66a405d9c4806a55cc8386f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3835eeee87b422bb3c32248602f145c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:33.011661 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac3d3c-9951-4bc9-b7b8-eba1747743c7 tempest-VolumeQuotaClassesTest-671961166 tempest-VolumeQuotaClassesTest-671961166-project-admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:33.011991 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac3d3c-9951-4bc9-b7b8-eba1747743c7 tempest-VolumeQuotaClassesTest-671961166 tempest-VolumeQuotaClassesTest-671961166-project-admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:33.012323 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac3d3c-9951-4bc9-b7b8-eba1747743c7 tempest-VolumeQuotaClassesTest-671961166 tempest-VolumeQuotaClassesTest-671961166-project-admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:33.013127 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ac3d3c-9951-4bc9-b7b8-eba1747743c7 tempest-VolumeQuotaClassesTest-671961166 tempest-VolumeQuotaClassesTest-671961166-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:41:33.013484 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ac3d3c-9951-4bc9-b7b8-eba1747743c7 tempest-VolumeQuotaClassesTest-671961166 tempest-VolumeQuotaClassesTest-671961166-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:33.015447 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ac3d3c-9951-4bc9-b7b8-eba1747743c7 tempest-VolumeQuotaClassesTest-671961166 tempest-VolumeQuotaClassesTest-671961166-project-admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:33.029402 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1333/2626] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:41:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:41:33.093098 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf9a327-41fa-4ffb-968f-247576ee1ab6 tempest-VolumeQuotaClassesTest-671961166 tempest-VolumeQuotaClassesTest-671961166-project-admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:33.106952 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf9a327-41fa-4ffb-968f-247576ee1ab6 tempest-VolumeQuotaClassesTest-671961166 tempest-VolumeQuotaClassesTest-671961166-project-admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:33.113509 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caee077-23e7-43aa-8982-9fe19d63eced tempest-VolumeQuotaClassesTest-671961166 tempest-VolumeQuotaClassesTest-671961166-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42a28a66a405d9c4806a55cc8386f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3835eeee87b422bb3c32248602f145c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:33.114599 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caee077-23e7-43aa-8982-9fe19d63eced tempest-VolumeQuotaClassesTest-671961166 tempest-VolumeQuotaClassesTest-671961166-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:33.115044 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caee077-23e7-43aa-8982-9fe19d63eced tempest-VolumeQuotaClassesTest-671961166 tempest-VolumeQuotaClassesTest-671961166-project-admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:33.115463 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caee077-23e7-43aa-8982-9fe19d63eced tempest-VolumeQuotaClassesTest-671961166 tempest-VolumeQuotaClassesTest-671961166-project-admin] PATH_INFO: `/v3/projects/7a6af418569249d598c6c2a2a8c8931d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:33.117284 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caee077-23e7-43aa-8982-9fe19d63eced tempest-VolumeQuotaClassesTest-671961166 tempest-VolumeQuotaClassesTest-671961166-project-admin] RBAC: Authorizing `identity:delete_project(project_id=7a6af418569249d598c6c2a2a8c8931d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:33.119502 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caee077-23e7-43aa-8982-9fe19d63eced tempest-VolumeQuotaClassesTest-671961166 tempest-VolumeQuotaClassesTest-671961166-project-admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:33.194849 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-4caee077-23e7-43aa-8982-9fe19d63eced tempest-VolumeQuotaClassesTest-671961166 tempest-VolumeQuotaClassesTest-671961166-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a6af418569249d598c6c2a2a8c8931d', 'request_id': 'req-4caee077-23e7-43aa-8982-9fe19d63eced'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:33.206116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-4caee077-23e7-43aa-8982-9fe19d63eced tempest-VolumeQuotaClassesTest-671961166 tempest-VolumeQuotaClassesTest-671961166-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a6af418569249d598c6c2a2a8c8931d', 'request_id': 'req-4caee077-23e7-43aa-8982-9fe19d63eced'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:33.216515 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-4caee077-23e7-43aa-8982-9fe19d63eced tempest-VolumeQuotaClassesTest-671961166 tempest-VolumeQuotaClassesTest-671961166-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a6af418569249d598c6c2a2a8c8931d', 'request_id': 'req-4caee077-23e7-43aa-8982-9fe19d63eced'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:33.235054 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-4caee077-23e7-43aa-8982-9fe19d63eced tempest-VolumeQuotaClassesTest-671961166 tempest-VolumeQuotaClassesTest-671961166-project-admin] The token cache is being invalidate because project 7a6af418569249d598c6c2a2a8c8931d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:41:33.235054 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-4caee077-23e7-43aa-8982-9fe19d63eced tempest-VolumeQuotaClassesTest-671961166 tempest-VolumeQuotaClassesTest-671961166-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:33.236175 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-4caee077-23e7-43aa-8982-9fe19d63eced tempest-VolumeQuotaClassesTest-671961166 tempest-VolumeQuotaClassesTest-671961166-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:33.238223 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1294/2627] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:41:33 2022] DELETE /identity/v3/projects/7a6af418569249d598c6c2a2a8c8931d => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:41:33.521370 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac3d3c-9951-4bc9-b7b8-eba1747743c7 tempest-VolumeQuotaClassesTest-671961166 tempest-VolumeQuotaClassesTest-671961166-project-admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:33.523850 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-34ac3d3c-9951-4bc9-b7b8-eba1747743c7 tempest-VolumeQuotaClassesTest-671961166 tempest-VolumeQuotaClassesTest-671961166-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:33.629320 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac3d3c-9951-4bc9-b7b8-eba1747743c7 tempest-VolumeQuotaClassesTest-671961166 tempest-VolumeQuotaClassesTest-671961166-project-admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:33.663175 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:41:33.707919 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27659120-eaf9-4ab2-8fa0-b81e19771a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:33.708895 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27659120-eaf9-4ab2-8fa0-b81e19771a8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:33.709226 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27659120-eaf9-4ab2-8fa0-b81e19771a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:33.709567 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27659120-eaf9-4ab2-8fa0-b81e19771a8e admin admin] PATH_INFO: `/v3/users/f40b58a0f7294c32bfb18fa672ec9279` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:33.711361 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27659120-eaf9-4ab2-8fa0-b81e19771a8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f40b58a0f7294c32bfb18fa672ec9279)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:33.713932 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27659120-eaf9-4ab2-8fa0-b81e19771a8e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:33.781759 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-27659120-eaf9-4ab2-8fa0-b81e19771a8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f40b58a0f7294c32bfb18fa672ec9279'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:33.811359 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-27659120-eaf9-4ab2-8fa0-b81e19771a8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f40b58a0f7294c32bfb18fa672ec9279'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:33.812416 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-27659120-eaf9-4ab2-8fa0-b81e19771a8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f40b58a0f7294c32bfb18fa672ec9279'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:33.813598 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-27659120-eaf9-4ab2-8fa0-b81e19771a8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f40b58a0f7294c32bfb18fa672ec9279'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:33.826093 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-27659120-eaf9-4ab2-8fa0-b81e19771a8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f40b58a0f7294c32bfb18fa672ec9279'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:33.850860 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1334/2628] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:41:33 2022] DELETE /identity/v3/users/f40b58a0f7294c32bfb18fa672ec9279 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:41:33.857616 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ef74f-3037-42bb-a410-230f3470bc4c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:33.858398 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ef74f-3037-42bb-a410-230f3470bc4c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:33.858777 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ef74f-3037-42bb-a410-230f3470bc4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:33.879830 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-e63ef74f-3037-42bb-a410-230f3470bc4c None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:41:33.881437 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e63ef74f-3037-42bb-a410-230f3470bc4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:33.966377 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1295/2629] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:41:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:41:33.979976 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27659120-eaf9-4ab2-8fa0-b81e19771a8e admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:33.981845 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-27659120-eaf9-4ab2-8fa0-b81e19771a8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:34.076215 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27659120-eaf9-4ab2-8fa0-b81e19771a8e admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:34.081167 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4f365d-4e1d-418d-bd99-f7948aefc82a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:34.081921 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f365d-4e1d-418d-bd99-f7948aefc82a service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:34.082187 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f365d-4e1d-418d-bd99-f7948aefc82a service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:34.082467 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f365d-4e1d-418d-bd99-f7948aefc82a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:34.095209 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4f365d-4e1d-418d-bd99-f7948aefc82a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:34.097122 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4f365d-4e1d-418d-bd99-f7948aefc82a service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:34.109498 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1335/2630] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:41:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:41:34.508254 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63ef74f-3037-42bb-a410-230f3470bc4c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:34.510188 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e63ef74f-3037-42bb-a410-230f3470bc4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:34.527060 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63ef74f-3037-42bb-a410-230f3470bc4c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:34.537053 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5fe81-1c94-49d1-b732-a7a71d44eda2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:34.538747 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5fe81-1c94-49d1-b732-a7a71d44eda2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:34.539339 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5fe81-1c94-49d1-b732-a7a71d44eda2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:34.539940 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5fe81-1c94-49d1-b732-a7a71d44eda2 admin admin] PATH_INFO: `/v3/projects/10e4801c1ef1487d90e1b71b88475355` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:34.542547 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed5fe81-1c94-49d1-b732-a7a71d44eda2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10e4801c1ef1487d90e1b71b88475355)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:34.546541 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed5fe81-1c94-49d1-b732-a7a71d44eda2 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:34.615524 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-bed5fe81-1c94-49d1-b732-a7a71d44eda2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10e4801c1ef1487d90e1b71b88475355', 'request_id': 'req-bed5fe81-1c94-49d1-b732-a7a71d44eda2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:34.629875 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-bed5fe81-1c94-49d1-b732-a7a71d44eda2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10e4801c1ef1487d90e1b71b88475355', 'request_id': 'req-bed5fe81-1c94-49d1-b732-a7a71d44eda2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:34.640224 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-bed5fe81-1c94-49d1-b732-a7a71d44eda2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10e4801c1ef1487d90e1b71b88475355', 'request_id': 'req-bed5fe81-1c94-49d1-b732-a7a71d44eda2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:34.657828 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-bed5fe81-1c94-49d1-b732-a7a71d44eda2 admin admin] The token cache is being invalidate because project 10e4801c1ef1487d90e1b71b88475355 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:41:34.658011 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-bed5fe81-1c94-49d1-b732-a7a71d44eda2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:34.659339 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-bed5fe81-1c94-49d1-b732-a7a71d44eda2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:34.666219 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1296/2631] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:41:34 2022] DELETE /identity/v3/projects/10e4801c1ef1487d90e1b71b88475355 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:41:34.668174 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4f365d-4e1d-418d-bd99-f7948aefc82a service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:34.685745 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4f365d-4e1d-418d-bd99-f7948aefc82a service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:34.754852 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:41:34.771568 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516c03cf-53f5-43c9-9437-d87d77f7373e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:34.772516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c03cf-53f5-43c9-9437-d87d77f7373e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:34.772800 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c03cf-53f5-43c9-9437-d87d77f7373e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:34.773086 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c03cf-53f5-43c9-9437-d87d77f7373e admin admin] PATH_INFO: `/v3/users/64a42a28a66a405d9c4806a55cc8386f` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:34.774697 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516c03cf-53f5-43c9-9437-d87d77f7373e admin admin] RBAC: Authorizing `identity:delete_user(user_id=64a42a28a66a405d9c4806a55cc8386f)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:34.776703 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516c03cf-53f5-43c9-9437-d87d77f7373e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:34.863377 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-516c03cf-53f5-43c9-9437-d87d77f7373e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64a42a28a66a405d9c4806a55cc8386f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:34.889486 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-516c03cf-53f5-43c9-9437-d87d77f7373e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64a42a28a66a405d9c4806a55cc8386f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:34.890727 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-516c03cf-53f5-43c9-9437-d87d77f7373e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64a42a28a66a405d9c4806a55cc8386f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:34.891740 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-516c03cf-53f5-43c9-9437-d87d77f7373e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64a42a28a66a405d9c4806a55cc8386f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:34.904792 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-516c03cf-53f5-43c9-9437-d87d77f7373e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64a42a28a66a405d9c4806a55cc8386f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:34.921858 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1336/2632] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:41:34 2022] DELETE /identity/v3/users/64a42a28a66a405d9c4806a55cc8386f => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:41:35.287731 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5fe81-1c94-49d1-b732-a7a71d44eda2 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:35.289565 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-bed5fe81-1c94-49d1-b732-a7a71d44eda2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:35.300944 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:41:35.307702 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5fe81-1c94-49d1-b732-a7a71d44eda2 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:35.416126 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15f7cb-be30-4173-aa47-025faa5f775a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:35.417721 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15f7cb-be30-4173-aa47-025faa5f775a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:35.418123 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15f7cb-be30-4173-aa47-025faa5f775a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:35.418499 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15f7cb-be30-4173-aa47-025faa5f775a admin admin] PATH_INFO: `/v3/projects/a3835eeee87b422bb3c32248602f145c` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:35.420602 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf15f7cb-be30-4173-aa47-025faa5f775a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3835eeee87b422bb3c32248602f145c)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:35.422875 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf15f7cb-be30-4173-aa47-025faa5f775a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:35.491891 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-bf15f7cb-be30-4173-aa47-025faa5f775a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3835eeee87b422bb3c32248602f145c', 'request_id': 'req-bf15f7cb-be30-4173-aa47-025faa5f775a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:35.502695 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-bf15f7cb-be30-4173-aa47-025faa5f775a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3835eeee87b422bb3c32248602f145c', 'request_id': 'req-bf15f7cb-be30-4173-aa47-025faa5f775a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:35.511330 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-bf15f7cb-be30-4173-aa47-025faa5f775a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3835eeee87b422bb3c32248602f145c', 'request_id': 'req-bf15f7cb-be30-4173-aa47-025faa5f775a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:35.530879 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-bf15f7cb-be30-4173-aa47-025faa5f775a admin admin] The token cache is being invalidate because project a3835eeee87b422bb3c32248602f145c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:41:35.531344 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-bf15f7cb-be30-4173-aa47-025faa5f775a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:35.532524 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-bf15f7cb-be30-4173-aa47-025faa5f775a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:35.535476 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1297/2633] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:41:35 2022] DELETE /identity/v3/projects/a3835eeee87b422bb3c32248602f145c => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:41:35.541167 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900dfd0d-6d6e-4f99-b36b-981dd801eae1 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:35.541529 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900dfd0d-6d6e-4f99-b36b-981dd801eae1 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:35.541876 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900dfd0d-6d6e-4f99-b36b-981dd801eae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:35.563531 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-900dfd0d-6d6e-4f99-b36b-981dd801eae1 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:41:35.566080 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-900dfd0d-6d6e-4f99-b36b-981dd801eae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:35.665808 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1337/2634] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:41:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:41:35.674900 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab44bd6-8412-42cb-a410-b7a445a1b084 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:35.675250 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab44bd6-8412-42cb-a410-b7a445a1b084 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:35.675466 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab44bd6-8412-42cb-a410-b7a445a1b084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:35.714515 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-5ab44bd6-8412-42cb-a410-b7a445a1b084 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:41:35.717095 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5ab44bd6-8412-42cb-a410-b7a445a1b084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:35.732059 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1298/2635] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:41:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:41:35.738092 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900dfd0d-6d6e-4f99-b36b-981dd801eae1 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:35.763747 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900dfd0d-6d6e-4f99-b36b-981dd801eae1 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:35.771037 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6cd5f8-3a84-4954-8673-e608912eefd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:35.772184 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6cd5f8-3a84-4954-8673-e608912eefd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:35.772531 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6cd5f8-3a84-4954-8673-e608912eefd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:35.773096 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6cd5f8-3a84-4954-8673-e608912eefd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:35.774080 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6cd5f8-3a84-4954-8673-e608912eefd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:41:35.774446 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6cd5f8-3a84-4954-8673-e608912eefd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:35.776591 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6cd5f8-3a84-4954-8673-e608912eefd5 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:35.793387 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1338/2636] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:41:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:41:35.801036 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab44bd6-8412-42cb-a410-b7a445a1b084 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:35.813457 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab44bd6-8412-42cb-a410-b7a445a1b084 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:35.822415 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d2bf95-66dc-4f88-b636-4eab47285b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:35.823856 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d2bf95-66dc-4f88-b636-4eab47285b6e admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:35.824367 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d2bf95-66dc-4f88-b636-4eab47285b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:35.824862 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d2bf95-66dc-4f88-b636-4eab47285b6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:35.825970 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d2bf95-66dc-4f88-b636-4eab47285b6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:35.828418 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d2bf95-66dc-4f88-b636-4eab47285b6e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:35.875253 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1299/2637] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:41:35 2022] POST /identity/v3/projects => generated 399 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:41:35.883381 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6cd5f8-3a84-4954-8673-e608912eefd5 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:35.897147 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6cd5f8-3a84-4954-8673-e608912eefd5 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:35.995551 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:41:36.013249 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5d7a5-a946-4d27-88e9-2b12f83bee63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:36.014745 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5d7a5-a946-4d27-88e9-2b12f83bee63 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:36.015397 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5d7a5-a946-4d27-88e9-2b12f83bee63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:36.015907 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5d7a5-a946-4d27-88e9-2b12f83bee63 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:36.017432 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab5d7a5-a946-4d27-88e9-2b12f83bee63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:36.024776 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab5d7a5-a946-4d27-88e9-2b12f83bee63 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:36.060240 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1339/2638] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:41:35 2022] POST /identity/v3/users => generated 300 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:41:36.067773 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d2bf95-66dc-4f88-b636-4eab47285b6e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:36.090758 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d2bf95-66dc-4f88-b636-4eab47285b6e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:36.101419 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fe01fc-90ac-4d05-a412-7c4959a11229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:36.102749 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe01fc-90ac-4d05-a412-7c4959a11229 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:36.103259 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe01fc-90ac-4d05-a412-7c4959a11229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:36.103757 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe01fc-90ac-4d05-a412-7c4959a11229 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:36.105178 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fe01fc-90ac-4d05-a412-7c4959a11229 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:36.108577 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fe01fc-90ac-4d05-a412-7c4959a11229 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:36.127986 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1300/2639] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:41:36.132458 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5d7a5-a946-4d27-88e9-2b12f83bee63 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:36.150985 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5d7a5-a946-4d27-88e9-2b12f83bee63 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:36.157625 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bdeb4f-59fa-48bc-8b0d-b327db4f8054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:36.158585 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bdeb4f-59fa-48bc-8b0d-b327db4f8054 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:36.159138 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bdeb4f-59fa-48bc-8b0d-b327db4f8054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:36.159653 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bdeb4f-59fa-48bc-8b0d-b327db4f8054 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:36.161047 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bdeb4f-59fa-48bc-8b0d-b327db4f8054 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:36.162921 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bdeb4f-59fa-48bc-8b0d-b327db4f8054 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:36.176562 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1340/2640] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:41:36.183457 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fe01fc-90ac-4d05-a412-7c4959a11229 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:36.204445 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fe01fc-90ac-4d05-a412-7c4959a11229 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:36.212650 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3f1531-3564-453f-bb58-2c532164d7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:36.214016 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f1531-3564-453f-bb58-2c532164d7cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:36.214526 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f1531-3564-453f-bb58-2c532164d7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:36.215043 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f1531-3564-453f-bb58-2c532164d7cb admin admin] PATH_INFO: `/v3/projects/34a67094c3b04efd9048f2c6d5f7f676/users/8584d9f086f249708601db610c9028ad/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:36.245362 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3f1531-3564-453f-bb58-2c532164d7cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=34a67094c3b04efd9048f2c6d5f7f676, user_id=8584d9f086f249708601db610c9028ad, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:36.251421 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3f1531-3564-453f-bb58-2c532164d7cb admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:36.277369 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1301/2641] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:41:36 2022] PUT /identity/v3/projects/34a67094c3b04efd9048f2c6d5f7f676/users/8584d9f086f249708601db610c9028ad/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:41:36.290507 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5622aa9-be54-4e53-b544-1c584167f1c0 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:36.291077 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5622aa9-be54-4e53-b544-1c584167f1c0 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:36.291616 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5622aa9-be54-4e53-b544-1c584167f1c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:36.330770 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-f5622aa9-be54-4e53-b544-1c584167f1c0 None None] MFA Rules not processed for user `8584d9f086f249708601db610c9028ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:41:36.332938 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f5622aa9-be54-4e53-b544-1c584167f1c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:36.451010 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1341/2642] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:41:36 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:41:36.459251 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3f1531-3564-453f-bb58-2c532164d7cb admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:36.509328 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:41:36.549498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3f1531-3564-453f-bb58-2c532164d7cb admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:36.555720 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222fe4cb-a3fb-428d-9133-914b496b71fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:36.556632 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222fe4cb-a3fb-428d-9133-914b496b71fe admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:36.557024 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222fe4cb-a3fb-428d-9133-914b496b71fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:36.557400 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222fe4cb-a3fb-428d-9133-914b496b71fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:36.558256 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222fe4cb-a3fb-428d-9133-914b496b71fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:36.560365 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222fe4cb-a3fb-428d-9133-914b496b71fe admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:36.596230 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1302/2643] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:41:36 2022] POST /identity/v3/projects => generated 401 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:41:36.602554 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5622aa9-be54-4e53-b544-1c584167f1c0 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:36.619957 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5622aa9-be54-4e53-b544-1c584167f1c0 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:36.731521 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdf55fd-9a06-4e7b-a2f3-b62b7b1d088b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:36.733011 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf55fd-9a06-4e7b-a2f3-b62b7b1d088b admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:36.733959 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf55fd-9a06-4e7b-a2f3-b62b7b1d088b admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:36.734148 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf55fd-9a06-4e7b-a2f3-b62b7b1d088b admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:36.736572 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdf55fd-9a06-4e7b-a2f3-b62b7b1d088b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:36.739111 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdf55fd-9a06-4e7b-a2f3-b62b7b1d088b admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:36.771901 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1342/2644] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:41:36 2022] POST /identity/v3/users => generated 307 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:41:36.776993 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222fe4cb-a3fb-428d-9133-914b496b71fe admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:36.791127 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222fe4cb-a3fb-428d-9133-914b496b71fe admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:36.806423 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4f1e08-b474-436e-bae7-469cf8b64f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:36.807503 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f1e08-b474-436e-bae7-469cf8b64f1f admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:36.807834 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f1e08-b474-436e-bae7-469cf8b64f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:36.808170 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f1e08-b474-436e-bae7-469cf8b64f1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:36.814030 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4f1e08-b474-436e-bae7-469cf8b64f1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:36.814030 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4f1e08-b474-436e-bae7-469cf8b64f1f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:36.831482 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1303/2645] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:41:36.836344 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdf55fd-9a06-4e7b-a2f3-b62b7b1d088b admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:36.852854 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdf55fd-9a06-4e7b-a2f3-b62b7b1d088b admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:36.858232 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee360e1c-14b4-4961-bcd4-c0220784e15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:36.859516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee360e1c-14b4-4961-bcd4-c0220784e15a admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:36.859516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee360e1c-14b4-4961-bcd4-c0220784e15a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:36.861585 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee360e1c-14b4-4961-bcd4-c0220784e15a admin admin] PATH_INFO: `/v3/projects/ec61ce2f1994485880ca0fe608c9bf45/users/cdf2b78218be4bb78712797dada90c38/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:36.883957 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee360e1c-14b4-4961-bcd4-c0220784e15a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec61ce2f1994485880ca0fe608c9bf45, user_id=cdf2b78218be4bb78712797dada90c38, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:36.889609 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee360e1c-14b4-4961-bcd4-c0220784e15a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:36.913906 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1343/2646] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:41:36 2022] PUT /identity/v3/projects/ec61ce2f1994485880ca0fe608c9bf45/users/cdf2b78218be4bb78712797dada90c38/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:41:36.918489 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4f1e08-b474-436e-bae7-469cf8b64f1f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:36.931803 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4f1e08-b474-436e-bae7-469cf8b64f1f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:37.032289 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ba00f-0826-4f48-8538-7d7a9fe6105a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:37.033343 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ba00f-0826-4f48-8538-7d7a9fe6105a admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:37.033931 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ba00f-0826-4f48-8538-7d7a9fe6105a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:37.034225 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ba00f-0826-4f48-8538-7d7a9fe6105a admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:37.035564 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283ba00f-0826-4f48-8538-7d7a9fe6105a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:37.038609 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283ba00f-0826-4f48-8538-7d7a9fe6105a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:37.056482 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1304/2647] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:41:37.063205 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee360e1c-14b4-4961-bcd4-c0220784e15a admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:37.082918 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee360e1c-14b4-4961-bcd4-c0220784e15a admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:37.091418 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047302be-8596-4a50-ab20-87ff47d1de9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:37.093022 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047302be-8596-4a50-ab20-87ff47d1de9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:37.093561 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047302be-8596-4a50-ab20-87ff47d1de9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:37.094090 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047302be-8596-4a50-ab20-87ff47d1de9b admin admin] PATH_INFO: `/v3/projects/ec61ce2f1994485880ca0fe608c9bf45/users/cdf2b78218be4bb78712797dada90c38/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:37.098671 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047302be-8596-4a50-ab20-87ff47d1de9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec61ce2f1994485880ca0fe608c9bf45, user_id=cdf2b78218be4bb78712797dada90c38, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:37.104050 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047302be-8596-4a50-ab20-87ff47d1de9b admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:37.125061 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1344/2648] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:41:37 2022] PUT /identity/v3/projects/ec61ce2f1994485880ca0fe608c9bf45/users/cdf2b78218be4bb78712797dada90c38/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:41:37.145673 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce19f9c-92f7-4bc4-83e9-ba15b001c27e None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:37.145995 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce19f9c-92f7-4bc4-83e9-ba15b001c27e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:37.146266 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce19f9c-92f7-4bc4-83e9-ba15b001c27e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:37.183069 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-5ce19f9c-92f7-4bc4-83e9-ba15b001c27e None None] MFA Rules not processed for user `cdf2b78218be4bb78712797dada90c38`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:41:37.184341 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5ce19f9c-92f7-4bc4-83e9-ba15b001c27e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:37.306140 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1305/2649] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:41:37 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:41:37.320674 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047302be-8596-4a50-ab20-87ff47d1de9b admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:37.323507 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-047302be-8596-4a50-ab20-87ff47d1de9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:37.348575 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:41:37.418059 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047302be-8596-4a50-ab20-87ff47d1de9b admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:37.423905 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6297a53b-a0d6-482b-8d8a-b5afea8f60ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:37.425516 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297a53b-a0d6-482b-8d8a-b5afea8f60ee service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:37.426027 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297a53b-a0d6-482b-8d8a-b5afea8f60ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:37.426477 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297a53b-a0d6-482b-8d8a-b5afea8f60ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:37.441598 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6297a53b-a0d6-482b-8d8a-b5afea8f60ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:37.443382 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6297a53b-a0d6-482b-8d8a-b5afea8f60ee service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:37.456987 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1345/2650] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:41:37 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:41:37.548281 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce19f9c-92f7-4bc4-83e9-ba15b001c27e None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:37.559764 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce19f9c-92f7-4bc4-83e9-ba15b001c27e None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:37.564298 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a9ff61-c317-47ea-8523-d1d89c344685 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:37.565110 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a9ff61-c317-47ea-8523-d1d89c344685 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:37.565418 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a9ff61-c317-47ea-8523-d1d89c344685 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:37.565737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a9ff61-c317-47ea-8523-d1d89c344685 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:37.593419 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:41:37.640524 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a9ff61-c317-47ea-8523-d1d89c344685 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:37.642041 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a9ff61-c317-47ea-8523-d1d89c344685 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:37.652290 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1306/2651] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:41:37 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:41:42.197922 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6297a53b-a0d6-482b-8d8a-b5afea8f60ee service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:42.328403 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6297a53b-a0d6-482b-8d8a-b5afea8f60ee service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:42.342871 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2693e2e8-49bc-40df-b24f-64e995a14576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:42.342871 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693e2e8-49bc-40df-b24f-64e995a14576 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:42.342871 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693e2e8-49bc-40df-b24f-64e995a14576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:42.342871 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693e2e8-49bc-40df-b24f-64e995a14576 admin admin] PATH_INFO: `/v3/users/8584d9f086f249708601db610c9028ad` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:42.343399 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2693e2e8-49bc-40df-b24f-64e995a14576 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8584d9f086f249708601db610c9028ad)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:42.346852 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2693e2e8-49bc-40df-b24f-64e995a14576 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:42.429905 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-2693e2e8-49bc-40df-b24f-64e995a14576 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8584d9f086f249708601db610c9028ad'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:42.454290 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-2693e2e8-49bc-40df-b24f-64e995a14576 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8584d9f086f249708601db610c9028ad'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:42.456148 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-2693e2e8-49bc-40df-b24f-64e995a14576 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8584d9f086f249708601db610c9028ad'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:42.457106 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-2693e2e8-49bc-40df-b24f-64e995a14576 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8584d9f086f249708601db610c9028ad'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:42.469605 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-2693e2e8-49bc-40df-b24f-64e995a14576 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8584d9f086f249708601db610c9028ad'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:42.494604 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1346/2652] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:41:42 2022] DELETE /identity/v3/users/8584d9f086f249708601db610c9028ad => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:41:42.499836 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ab54d-eef6-48c0-a90b-25020a89f078 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:42.500713 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ab54d-eef6-48c0-a90b-25020a89f078 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:42.500713 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ab54d-eef6-48c0-a90b-25020a89f078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:42.527752 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d86ab54d-eef6-48c0-a90b-25020a89f078 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:41:42.529183 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d86ab54d-eef6-48c0-a90b-25020a89f078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:42.621184 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1307/2653] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:41:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:41:42.635140 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2693e2e8-49bc-40df-b24f-64e995a14576 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:42.637394 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-2693e2e8-49bc-40df-b24f-64e995a14576 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:42.707583 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:41:42.732721 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2693e2e8-49bc-40df-b24f-64e995a14576 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:42.736691 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea899726-f1c4-48bc-8524-9bae39c5fcdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:42.737480 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea899726-f1c4-48bc-8524-9bae39c5fcdc service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:42.738021 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea899726-f1c4-48bc-8524-9bae39c5fcdc service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:42.738384 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea899726-f1c4-48bc-8524-9bae39c5fcdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:42.754067 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea899726-f1c4-48bc-8524-9bae39c5fcdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:42.756137 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea899726-f1c4-48bc-8524-9bae39c5fcdc service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:42.769628 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1347/2654] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:41:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:41:43.216751 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86ab54d-eef6-48c0-a90b-25020a89f078 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:43.219269 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d86ab54d-eef6-48c0-a90b-25020a89f078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:43.246349 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86ab54d-eef6-48c0-a90b-25020a89f078 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:43.254930 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73a70d-74b2-43ca-95d3-084797d11cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:43.256462 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73a70d-74b2-43ca-95d3-084797d11cce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:43.256952 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73a70d-74b2-43ca-95d3-084797d11cce admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:43.257426 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73a70d-74b2-43ca-95d3-084797d11cce admin admin] PATH_INFO: `/v3/projects/34a67094c3b04efd9048f2c6d5f7f676` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:43.260466 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f73a70d-74b2-43ca-95d3-084797d11cce admin admin] RBAC: Authorizing `identity:delete_project(project_id=34a67094c3b04efd9048f2c6d5f7f676)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:43.265323 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f73a70d-74b2-43ca-95d3-084797d11cce admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:43.355696 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3f73a70d-74b2-43ca-95d3-084797d11cce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34a67094c3b04efd9048f2c6d5f7f676', 'request_id': 'req-3f73a70d-74b2-43ca-95d3-084797d11cce'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:43.366678 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3f73a70d-74b2-43ca-95d3-084797d11cce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34a67094c3b04efd9048f2c6d5f7f676', 'request_id': 'req-3f73a70d-74b2-43ca-95d3-084797d11cce'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:43.374853 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3f73a70d-74b2-43ca-95d3-084797d11cce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34a67094c3b04efd9048f2c6d5f7f676', 'request_id': 'req-3f73a70d-74b2-43ca-95d3-084797d11cce'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:43.390973 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3f73a70d-74b2-43ca-95d3-084797d11cce admin admin] The token cache is being invalidate because project 34a67094c3b04efd9048f2c6d5f7f676 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:41:43.391276 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3f73a70d-74b2-43ca-95d3-084797d11cce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:43.392146 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3f73a70d-74b2-43ca-95d3-084797d11cce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:43.394501 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1308/2655] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:41:43 2022] DELETE /identity/v3/projects/34a67094c3b04efd9048f2c6d5f7f676 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:41:43.399617 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea899726-f1c4-48bc-8524-9bae39c5fcdc service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:43.414889 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea899726-f1c4-48bc-8524-9bae39c5fcdc service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:43.489738 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9fdf07-f97e-4caf-8687-f37afa778cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:43.490714 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9fdf07-f97e-4caf-8687-f37afa778cec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:43.491318 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9fdf07-f97e-4caf-8687-f37afa778cec admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:43.491639 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9fdf07-f97e-4caf-8687-f37afa778cec admin admin] PATH_INFO: `/v3/users/cdf2b78218be4bb78712797dada90c38` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:43.493036 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9fdf07-f97e-4caf-8687-f37afa778cec admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdf2b78218be4bb78712797dada90c38)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:43.496066 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9fdf07-f97e-4caf-8687-f37afa778cec admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:43.564087 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8f9fdf07-f97e-4caf-8687-f37afa778cec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdf2b78218be4bb78712797dada90c38'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:43.579501 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8f9fdf07-f97e-4caf-8687-f37afa778cec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdf2b78218be4bb78712797dada90c38'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:43.580638 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8f9fdf07-f97e-4caf-8687-f37afa778cec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdf2b78218be4bb78712797dada90c38'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:43.581302 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8f9fdf07-f97e-4caf-8687-f37afa778cec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdf2b78218be4bb78712797dada90c38'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:43.592258 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8f9fdf07-f97e-4caf-8687-f37afa778cec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdf2b78218be4bb78712797dada90c38'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:43.609239 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1348/2656] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:41:43 2022] DELETE /identity/v3/users/cdf2b78218be4bb78712797dada90c38 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:41:44.015030 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73a70d-74b2-43ca-95d3-084797d11cce admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:44.020455 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3f73a70d-74b2-43ca-95d3-084797d11cce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:44.041592 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73a70d-74b2-43ca-95d3-084797d11cce admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:44.119470 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:41:44.141819 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887cf9b9-89a8-42da-8b4e-68e755c116d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:44.142726 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887cf9b9-89a8-42da-8b4e-68e755c116d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:44.143017 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887cf9b9-89a8-42da-8b4e-68e755c116d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:44.143318 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887cf9b9-89a8-42da-8b4e-68e755c116d1 admin admin] PATH_INFO: `/v3/projects/ec61ce2f1994485880ca0fe608c9bf45` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:44.144677 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887cf9b9-89a8-42da-8b4e-68e755c116d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec61ce2f1994485880ca0fe608c9bf45)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:44.146738 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887cf9b9-89a8-42da-8b4e-68e755c116d1 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:44.209387 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-887cf9b9-89a8-42da-8b4e-68e755c116d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec61ce2f1994485880ca0fe608c9bf45', 'request_id': 'req-887cf9b9-89a8-42da-8b4e-68e755c116d1'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:44.225245 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-887cf9b9-89a8-42da-8b4e-68e755c116d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec61ce2f1994485880ca0fe608c9bf45', 'request_id': 'req-887cf9b9-89a8-42da-8b4e-68e755c116d1'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:44.232966 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-887cf9b9-89a8-42da-8b4e-68e755c116d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec61ce2f1994485880ca0fe608c9bf45', 'request_id': 'req-887cf9b9-89a8-42da-8b4e-68e755c116d1'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:44.251352 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-887cf9b9-89a8-42da-8b4e-68e755c116d1 admin admin] The token cache is being invalidate because project ec61ce2f1994485880ca0fe608c9bf45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:41:44.251751 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-887cf9b9-89a8-42da-8b4e-68e755c116d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:44.252538 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-887cf9b9-89a8-42da-8b4e-68e755c116d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:44.255211 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1309/2657] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:41:44 2022] DELETE /identity/v3/projects/ec61ce2f1994485880ca0fe608c9bf45 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:41:44.262226 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf76453-eff3-4d30-bc0f-26c8a22060a9 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:44.262585 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf76453-eff3-4d30-bc0f-26c8a22060a9 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:44.262965 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf76453-eff3-4d30-bc0f-26c8a22060a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:44.295181 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-cbf76453-eff3-4d30-bc0f-26c8a22060a9 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:41:44.297192 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-cbf76453-eff3-4d30-bc0f-26c8a22060a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:44.385981 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:41:44.397973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1349/2658] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:41:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:41:44.406361 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa81f05-9ea9-40e1-84b3-978305c5ccbd None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:44.406801 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa81f05-9ea9-40e1-84b3-978305c5ccbd None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:44.407178 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa81f05-9ea9-40e1-84b3-978305c5ccbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:44.431493 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-0fa81f05-9ea9-40e1-84b3-978305c5ccbd None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:41:44.432824 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0fa81f05-9ea9-40e1-84b3-978305c5ccbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:44.442729 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1310/2659] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:41:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:41:44.448293 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf76453-eff3-4d30-bc0f-26c8a22060a9 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:44.467710 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf76453-eff3-4d30-bc0f-26c8a22060a9 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:44.480861 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09996049-a7da-4a75-aed6-3134562f0407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:44.481982 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09996049-a7da-4a75-aed6-3134562f0407 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:44.482436 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09996049-a7da-4a75-aed6-3134562f0407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:44.482887 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09996049-a7da-4a75-aed6-3134562f0407 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:44.484200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09996049-a7da-4a75-aed6-3134562f0407 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:41:44.484748 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09996049-a7da-4a75-aed6-3134562f0407 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:44.487970 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09996049-a7da-4a75-aed6-3134562f0407 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:44.513793 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1350/2660] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:41:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:41:44.516879 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa81f05-9ea9-40e1-84b3-978305c5ccbd None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:44.531431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa81f05-9ea9-40e1-84b3-978305c5ccbd None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:44.540141 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fee16d-0f6c-4cf1-b627-d12e4b587c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:44.541217 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fee16d-0f6c-4cf1-b627-d12e4b587c07 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:44.541788 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fee16d-0f6c-4cf1-b627-d12e4b587c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:44.542290 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fee16d-0f6c-4cf1-b627-d12e4b587c07 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:44.543629 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fee16d-0f6c-4cf1-b627-d12e4b587c07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:44.547540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fee16d-0f6c-4cf1-b627-d12e4b587c07 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:44.592140 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1311/2661] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:41:44 2022] POST /identity/v3/projects => generated 383 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:41:44.596899 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09996049-a7da-4a75-aed6-3134562f0407 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:44.609773 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09996049-a7da-4a75-aed6-3134562f0407 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:44.698703 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f91a86-31ff-4127-9ee8-f9b8b2a10d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:44.699436 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f91a86-31ff-4127-9ee8-f9b8b2a10d90 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:44.699744 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f91a86-31ff-4127-9ee8-f9b8b2a10d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:44.700064 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f91a86-31ff-4127-9ee8-f9b8b2a10d90 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:44.700908 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f91a86-31ff-4127-9ee8-f9b8b2a10d90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:44.703846 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f91a86-31ff-4127-9ee8-f9b8b2a10d90 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:44.743259 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1351/2662] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:41:44 2022] POST /identity/v3/users => generated 292 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:41:44.748275 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fee16d-0f6c-4cf1-b627-d12e4b587c07 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:44.761560 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fee16d-0f6c-4cf1-b627-d12e4b587c07 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:44.770575 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4e85f4-4848-41cd-aa7e-f6e71542efac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:44.771760 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4e85f4-4848-41cd-aa7e-f6e71542efac admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:44.772247 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4e85f4-4848-41cd-aa7e-f6e71542efac admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:44.772730 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4e85f4-4848-41cd-aa7e-f6e71542efac admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:44.774004 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4e85f4-4848-41cd-aa7e-f6e71542efac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:44.776694 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4e85f4-4848-41cd-aa7e-f6e71542efac admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:44.796474 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1312/2663] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:41:44 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:41:44.800861 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f91a86-31ff-4127-9ee8-f9b8b2a10d90 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:44.815410 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f91a86-31ff-4127-9ee8-f9b8b2a10d90 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:44.822139 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec9f24b-1356-45a8-835e-2eddc73bb08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:44.822891 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec9f24b-1356-45a8-835e-2eddc73bb08c admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:44.823284 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec9f24b-1356-45a8-835e-2eddc73bb08c admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:44.823622 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec9f24b-1356-45a8-835e-2eddc73bb08c admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:44.824417 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec9f24b-1356-45a8-835e-2eddc73bb08c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:44.826219 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec9f24b-1356-45a8-835e-2eddc73bb08c admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:44.848001 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1352/2664] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:41:44 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:41:44.850267 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4e85f4-4848-41cd-aa7e-f6e71542efac admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:44.865523 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4e85f4-4848-41cd-aa7e-f6e71542efac admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:44.874223 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5163c1-c0cd-4a20-9de9-f6e66963b35a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:44.875319 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5163c1-c0cd-4a20-9de9-f6e66963b35a admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:44.875703 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5163c1-c0cd-4a20-9de9-f6e66963b35a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:44.876082 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5163c1-c0cd-4a20-9de9-f6e66963b35a admin admin] PATH_INFO: `/v3/projects/6afb73f478604c4e979ca222eadfffa9/users/a1cdafc4351943f097cc38d543c279a6/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:44.895289 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5163c1-c0cd-4a20-9de9-f6e66963b35a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6afb73f478604c4e979ca222eadfffa9, user_id=a1cdafc4351943f097cc38d543c279a6, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:44.898359 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5163c1-c0cd-4a20-9de9-f6e66963b35a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:44.915701 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1313/2665] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:41:44 2022] PUT /identity/v3/projects/6afb73f478604c4e979ca222eadfffa9/users/a1cdafc4351943f097cc38d543c279a6/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:41:44.928242 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36cd572-c333-4c69-a18e-33fab56f6eeb None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:44.928699 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36cd572-c333-4c69-a18e-33fab56f6eeb None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:44.929045 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36cd572-c333-4c69-a18e-33fab56f6eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:44.961763 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-c36cd572-c333-4c69-a18e-33fab56f6eeb None None] MFA Rules not processed for user `a1cdafc4351943f097cc38d543c279a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:41:44.963021 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c36cd572-c333-4c69-a18e-33fab56f6eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:45.081510 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1353/2666] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:41:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:41:45.087868 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5163c1-c0cd-4a20-9de9-f6e66963b35a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:45.201413 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5163c1-c0cd-4a20-9de9-f6e66963b35a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:45.210246 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba97d9-dbee-457e-bcdb-8d113771cd80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:45.211412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba97d9-dbee-457e-bcdb-8d113771cd80 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:45.211920 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba97d9-dbee-457e-bcdb-8d113771cd80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:45.214177 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba97d9-dbee-457e-bcdb-8d113771cd80 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:45.215052 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ba97d9-dbee-457e-bcdb-8d113771cd80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:45.219130 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ba97d9-dbee-457e-bcdb-8d113771cd80 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:45.253826 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:41:45.270510 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1314/2667] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:41:45 2022] POST /identity/v3/projects => generated 383 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:41:45.275340 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36cd572-c333-4c69-a18e-33fab56f6eeb None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:45.294497 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36cd572-c333-4c69-a18e-33fab56f6eeb None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:45.383744 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:41:45.398473 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d218d3-ee6f-4c7f-a2a5-c804f145e3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:45.399589 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d218d3-ee6f-4c7f-a2a5-c804f145e3eb admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:45.400041 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d218d3-ee6f-4c7f-a2a5-c804f145e3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:45.400505 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d218d3-ee6f-4c7f-a2a5-c804f145e3eb admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:45.401778 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d218d3-ee6f-4c7f-a2a5-c804f145e3eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:45.404980 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d218d3-ee6f-4c7f-a2a5-c804f145e3eb admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:45.432363 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1354/2668] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:41:45 2022] POST /identity/v3/users => generated 298 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:41:45.437757 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba97d9-dbee-457e-bcdb-8d113771cd80 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:45.449497 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba97d9-dbee-457e-bcdb-8d113771cd80 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:45.454702 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c6d8f2-645e-4b75-aa87-4f4add7e2302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:45.455717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c6d8f2-645e-4b75-aa87-4f4add7e2302 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:45.456026 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c6d8f2-645e-4b75-aa87-4f4add7e2302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:45.456314 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c6d8f2-645e-4b75-aa87-4f4add7e2302 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:45.457173 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c6d8f2-645e-4b75-aa87-4f4add7e2302 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:45.458941 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c6d8f2-645e-4b75-aa87-4f4add7e2302 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:45.470993 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1315/2669] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:41:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:41:45.477289 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d218d3-ee6f-4c7f-a2a5-c804f145e3eb admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:45.490345 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d218d3-ee6f-4c7f-a2a5-c804f145e3eb admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:45.496513 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc31c8-56d9-4703-a78e-e94f6c633506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:45.497486 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc31c8-56d9-4703-a78e-e94f6c633506 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:45.497930 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc31c8-56d9-4703-a78e-e94f6c633506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:45.498302 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc31c8-56d9-4703-a78e-e94f6c633506 admin admin] PATH_INFO: `/v3/projects/2529d92d9d934d6ca28eec52bc87e41a/users/baca5144ec7748e8ba43a80e888a6a83/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:45.519453 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dc31c8-56d9-4703-a78e-e94f6c633506 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2529d92d9d934d6ca28eec52bc87e41a, user_id=baca5144ec7748e8ba43a80e888a6a83, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:45.522818 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dc31c8-56d9-4703-a78e-e94f6c633506 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:45.540967 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1355/2670] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:41:45 2022] PUT /identity/v3/projects/2529d92d9d934d6ca28eec52bc87e41a/users/baca5144ec7748e8ba43a80e888a6a83/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:41:45.546834 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c6d8f2-645e-4b75-aa87-4f4add7e2302 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:45.559158 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c6d8f2-645e-4b75-aa87-4f4add7e2302 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:45.650096 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688810c9-cbc8-45d0-a4dc-9d423e2b2902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:45.650929 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688810c9-cbc8-45d0-a4dc-9d423e2b2902 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:45.651310 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688810c9-cbc8-45d0-a4dc-9d423e2b2902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:45.651693 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688810c9-cbc8-45d0-a4dc-9d423e2b2902 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:45.652547 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688810c9-cbc8-45d0-a4dc-9d423e2b2902 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:45.654464 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688810c9-cbc8-45d0-a4dc-9d423e2b2902 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:45.666837 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1316/2671] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:41:45 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:41:45.671950 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc31c8-56d9-4703-a78e-e94f6c633506 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:45.684644 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc31c8-56d9-4703-a78e-e94f6c633506 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:45.690063 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce9c4be-25ad-44f6-ad99-a2ceed3f897c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:45.691433 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce9c4be-25ad-44f6-ad99-a2ceed3f897c admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:45.691862 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce9c4be-25ad-44f6-ad99-a2ceed3f897c admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:45.692251 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce9c4be-25ad-44f6-ad99-a2ceed3f897c admin admin] PATH_INFO: `/v3/projects/2529d92d9d934d6ca28eec52bc87e41a/users/baca5144ec7748e8ba43a80e888a6a83/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:45.695772 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce9c4be-25ad-44f6-ad99-a2ceed3f897c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2529d92d9d934d6ca28eec52bc87e41a, user_id=baca5144ec7748e8ba43a80e888a6a83, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:45.700747 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce9c4be-25ad-44f6-ad99-a2ceed3f897c admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:45.720936 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1356/2672] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:41:45 2022] PUT /identity/v3/projects/2529d92d9d934d6ca28eec52bc87e41a/users/baca5144ec7748e8ba43a80e888a6a83/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:41:45.734214 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e6f82-9829-4c4e-8a11-a97edad750c7 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:45.734534 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e6f82-9829-4c4e-8a11-a97edad750c7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:45.734824 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e6f82-9829-4c4e-8a11-a97edad750c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:45.767600 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-083e6f82-9829-4c4e-8a11-a97edad750c7 None None] MFA Rules not processed for user `baca5144ec7748e8ba43a80e888a6a83`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:41:45.769081 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-083e6f82-9829-4c4e-8a11-a97edad750c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:45.876400 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1317/2673] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:41:45 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:41:45.891678 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce9c4be-25ad-44f6-ad99-a2ceed3f897c admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:45.893970 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3ce9c4be-25ad-44f6-ad99-a2ceed3f897c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:45.993397 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce9c4be-25ad-44f6-ad99-a2ceed3f897c admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:45.999435 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95300a4f-be4d-4ff9-a93f-e18d4eb03148 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:46.000551 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95300a4f-be4d-4ff9-a93f-e18d4eb03148 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:46.001033 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95300a4f-be4d-4ff9-a93f-e18d4eb03148 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:46.001525 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95300a4f-be4d-4ff9-a93f-e18d4eb03148 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:46.019527 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95300a4f-be4d-4ff9-a93f-e18d4eb03148 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:46.022710 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95300a4f-be4d-4ff9-a93f-e18d4eb03148 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:46.042531 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1357/2674] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:41:45 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:41:46.194094 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e6f82-9829-4c4e-8a11-a97edad750c7 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:46.206254 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e6f82-9829-4c4e-8a11-a97edad750c7 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:46.212298 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749478a9-5b66-47d2-af5e-68b4b53031d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:46.213425 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749478a9-5b66-47d2-af5e-68b4b53031d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:46.214003 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749478a9-5b66-47d2-af5e-68b4b53031d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:46.214688 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749478a9-5b66-47d2-af5e-68b4b53031d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:46.330597 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749478a9-5b66-47d2-af5e-68b4b53031d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:46.333479 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749478a9-5b66-47d2-af5e-68b4b53031d3 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:46.350929 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1318/2675] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:41:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:41:50.288760 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95300a4f-be4d-4ff9-a93f-e18d4eb03148 service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:50.386260 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95300a4f-be4d-4ff9-a93f-e18d4eb03148 service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:50.391637 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d704211-79f3-49ec-b68c-91e79073a66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:50.392778 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d704211-79f3-49ec-b68c-91e79073a66b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:50.393172 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d704211-79f3-49ec-b68c-91e79073a66b admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:50.393594 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d704211-79f3-49ec-b68c-91e79073a66b admin admin] PATH_INFO: `/v3/users/a1cdafc4351943f097cc38d543c279a6` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:50.395105 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d704211-79f3-49ec-b68c-91e79073a66b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1cdafc4351943f097cc38d543c279a6)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:50.397147 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d704211-79f3-49ec-b68c-91e79073a66b admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:50.484832 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7d704211-79f3-49ec-b68c-91e79073a66b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1cdafc4351943f097cc38d543c279a6'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:50.495324 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:41:50.514447 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7d704211-79f3-49ec-b68c-91e79073a66b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1cdafc4351943f097cc38d543c279a6'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:50.515382 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7d704211-79f3-49ec-b68c-91e79073a66b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1cdafc4351943f097cc38d543c279a6'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:50.516663 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7d704211-79f3-49ec-b68c-91e79073a66b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1cdafc4351943f097cc38d543c279a6'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:50.527740 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7d704211-79f3-49ec-b68c-91e79073a66b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1cdafc4351943f097cc38d543c279a6'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:50.548131 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1358/2676] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:41:50 2022] DELETE /identity/v3/users/a1cdafc4351943f097cc38d543c279a6 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:41:50.555656 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e1f1c-7cb0-49ba-97e4-e03c41029abd None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:50.556414 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e1f1c-7cb0-49ba-97e4-e03c41029abd None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:50.556916 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e1f1c-7cb0-49ba-97e4-e03c41029abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:50.577061 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:41:50.586057 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a14e1f1c-7cb0-49ba-97e4-e03c41029abd None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:41:50.588246 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a14e1f1c-7cb0-49ba-97e4-e03c41029abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:50.688692 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1319/2677] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:41:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:41:50.702196 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d704211-79f3-49ec-b68c-91e79073a66b admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:50.704755 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7d704211-79f3-49ec-b68c-91e79073a66b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:50.800905 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d704211-79f3-49ec-b68c-91e79073a66b admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:50.805275 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31851cba-70fd-4c6b-8c21-7bfce165d968 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:50.805984 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31851cba-70fd-4c6b-8c21-7bfce165d968 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:50.806275 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31851cba-70fd-4c6b-8c21-7bfce165d968 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:50.806557 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31851cba-70fd-4c6b-8c21-7bfce165d968 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:50.820032 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31851cba-70fd-4c6b-8c21-7bfce165d968 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:50.821626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31851cba-70fd-4c6b-8c21-7bfce165d968 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:50.839031 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1359/2678] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:41:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:41:51.283860 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e1f1c-7cb0-49ba-97e4-e03c41029abd None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:51.286508 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a14e1f1c-7cb0-49ba-97e4-e03c41029abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:51.317105 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e1f1c-7cb0-49ba-97e4-e03c41029abd None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:51.326586 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73edaaba-ae55-4831-b128-03b6d73e8a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:51.328033 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73edaaba-ae55-4831-b128-03b6d73e8a61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:51.328547 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73edaaba-ae55-4831-b128-03b6d73e8a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:51.329050 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73edaaba-ae55-4831-b128-03b6d73e8a61 admin admin] PATH_INFO: `/v3/projects/6afb73f478604c4e979ca222eadfffa9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:51.333844 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73edaaba-ae55-4831-b128-03b6d73e8a61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6afb73f478604c4e979ca222eadfffa9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:51.336612 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73edaaba-ae55-4831-b128-03b6d73e8a61 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:51.422683 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-73edaaba-ae55-4831-b128-03b6d73e8a61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6afb73f478604c4e979ca222eadfffa9', 'request_id': 'req-73edaaba-ae55-4831-b128-03b6d73e8a61'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:51.435748 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-73edaaba-ae55-4831-b128-03b6d73e8a61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6afb73f478604c4e979ca222eadfffa9', 'request_id': 'req-73edaaba-ae55-4831-b128-03b6d73e8a61'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:51.447730 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-73edaaba-ae55-4831-b128-03b6d73e8a61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6afb73f478604c4e979ca222eadfffa9', 'request_id': 'req-73edaaba-ae55-4831-b128-03b6d73e8a61'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:51.476743 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-73edaaba-ae55-4831-b128-03b6d73e8a61 admin admin] The token cache is being invalidate because project 6afb73f478604c4e979ca222eadfffa9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:41:51.476743 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-73edaaba-ae55-4831-b128-03b6d73e8a61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:51.476743 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-73edaaba-ae55-4831-b128-03b6d73e8a61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:51.476743 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1320/2679] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:41:51 2022] DELETE /identity/v3/projects/6afb73f478604c4e979ca222eadfffa9 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:41:51.477297 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31851cba-70fd-4c6b-8c21-7bfce165d968 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:51.492998 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31851cba-70fd-4c6b-8c21-7bfce165d968 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:51.579268 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0d3804-e6cb-4c8e-9c61-a89ff4b02282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:51.580760 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0d3804-e6cb-4c8e-9c61-a89ff4b02282 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:51.581189 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0d3804-e6cb-4c8e-9c61-a89ff4b02282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:51.581670 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0d3804-e6cb-4c8e-9c61-a89ff4b02282 admin admin] PATH_INFO: `/v3/users/baca5144ec7748e8ba43a80e888a6a83` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:51.584118 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0d3804-e6cb-4c8e-9c61-a89ff4b02282 admin admin] RBAC: Authorizing `identity:delete_user(user_id=baca5144ec7748e8ba43a80e888a6a83)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:51.586344 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0d3804-e6cb-4c8e-9c61-a89ff4b02282 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:51.671282 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3a0d3804-e6cb-4c8e-9c61-a89ff4b02282 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baca5144ec7748e8ba43a80e888a6a83'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:51.694285 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3a0d3804-e6cb-4c8e-9c61-a89ff4b02282 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baca5144ec7748e8ba43a80e888a6a83'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:51.695424 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3a0d3804-e6cb-4c8e-9c61-a89ff4b02282 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baca5144ec7748e8ba43a80e888a6a83'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:51.696481 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3a0d3804-e6cb-4c8e-9c61-a89ff4b02282 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baca5144ec7748e8ba43a80e888a6a83'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:51.710968 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3a0d3804-e6cb-4c8e-9c61-a89ff4b02282 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baca5144ec7748e8ba43a80e888a6a83'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:51.714680 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:41:51.730955 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1360/2680] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:41:51 2022] DELETE /identity/v3/users/baca5144ec7748e8ba43a80e888a6a83 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:41:52.125664 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73edaaba-ae55-4831-b128-03b6d73e8a61 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:52.127765 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-73edaaba-ae55-4831-b128-03b6d73e8a61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:52.145337 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73edaaba-ae55-4831-b128-03b6d73e8a61 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:52.246347 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b53a1f-ea60-481e-b46f-659fc223c081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:52.247395 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b53a1f-ea60-481e-b46f-659fc223c081 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:52.247703 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b53a1f-ea60-481e-b46f-659fc223c081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:52.247983 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b53a1f-ea60-481e-b46f-659fc223c081 admin admin] PATH_INFO: `/v3/projects/2529d92d9d934d6ca28eec52bc87e41a` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:52.249579 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b53a1f-ea60-481e-b46f-659fc223c081 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2529d92d9d934d6ca28eec52bc87e41a)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:52.252406 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b53a1f-ea60-481e-b46f-659fc223c081 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:52.313118 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:41:52.341038 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-91b53a1f-ea60-481e-b46f-659fc223c081 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2529d92d9d934d6ca28eec52bc87e41a', 'request_id': 'req-91b53a1f-ea60-481e-b46f-659fc223c081'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:52.352971 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-91b53a1f-ea60-481e-b46f-659fc223c081 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2529d92d9d934d6ca28eec52bc87e41a', 'request_id': 'req-91b53a1f-ea60-481e-b46f-659fc223c081'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:52.361524 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-91b53a1f-ea60-481e-b46f-659fc223c081 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2529d92d9d934d6ca28eec52bc87e41a', 'request_id': 'req-91b53a1f-ea60-481e-b46f-659fc223c081'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:52.376517 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-91b53a1f-ea60-481e-b46f-659fc223c081 admin admin] The token cache is being invalidate because project 2529d92d9d934d6ca28eec52bc87e41a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:41:52.377028 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-91b53a1f-ea60-481e-b46f-659fc223c081 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:52.378170 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-91b53a1f-ea60-481e-b46f-659fc223c081 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:52.385179 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1321/2681] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:41:52 2022] DELETE /identity/v3/projects/2529d92d9d934d6ca28eec52bc87e41a => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:41:52.387865 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87affd2-56a0-40af-9650-73beaa57b708 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:52.388319 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87affd2-56a0-40af-9650-73beaa57b708 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:52.388658 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87affd2-56a0-40af-9650-73beaa57b708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:52.413044 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-f87affd2-56a0-40af-9650-73beaa57b708 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:41:52.414731 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f87affd2-56a0-40af-9650-73beaa57b708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:52.498624 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1361/2682] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:41:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:41:52.511339 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e3bc1-bc2e-4a9a-b2f0-3cebc7fc471e None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:52.512052 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e3bc1-bc2e-4a9a-b2f0-3cebc7fc471e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:52.512571 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e3bc1-bc2e-4a9a-b2f0-3cebc7fc471e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:52.542183 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-365e3bc1-bc2e-4a9a-b2f0-3cebc7fc471e None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:41:52.543568 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-365e3bc1-bc2e-4a9a-b2f0-3cebc7fc471e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:52.554327 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1322/2683] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:41:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:41:52.559354 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87affd2-56a0-40af-9650-73beaa57b708 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:52.571527 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87affd2-56a0-40af-9650-73beaa57b708 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:52.579298 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c99e37d-2383-4b49-8659-88b00b8a9e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:52.580578 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c99e37d-2383-4b49-8659-88b00b8a9e0f admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:52.581166 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c99e37d-2383-4b49-8659-88b00b8a9e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:52.581963 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c99e37d-2383-4b49-8659-88b00b8a9e0f admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:52.583365 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c99e37d-2383-4b49-8659-88b00b8a9e0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:41:52.583954 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c99e37d-2383-4b49-8659-88b00b8a9e0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:52.587157 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c99e37d-2383-4b49-8659-88b00b8a9e0f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:52.608120 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1362/2684] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:41:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:41:52.612901 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365e3bc1-bc2e-4a9a-b2f0-3cebc7fc471e None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:52.623608 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365e3bc1-bc2e-4a9a-b2f0-3cebc7fc471e None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:52.628933 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d25a98c-3ac2-4ee6-bea7-219a7fd8bd19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:52.629537 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25a98c-3ac2-4ee6-bea7-219a7fd8bd19 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:52.629898 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25a98c-3ac2-4ee6-bea7-219a7fd8bd19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:52.630246 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25a98c-3ac2-4ee6-bea7-219a7fd8bd19 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:52.631017 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d25a98c-3ac2-4ee6-bea7-219a7fd8bd19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:52.633418 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d25a98c-3ac2-4ee6-bea7-219a7fd8bd19 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:52.673528 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1323/2685] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:41:52 2022] POST /identity/v3/projects => generated 375 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:41:52.677732 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c99e37d-2383-4b49-8659-88b00b8a9e0f admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:52.689613 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c99e37d-2383-4b49-8659-88b00b8a9e0f admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:52.816828 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7cb529-8793-4ae3-afa1-f1bd9d5ae5f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:52.818228 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7cb529-8793-4ae3-afa1-f1bd9d5ae5f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:52.818889 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7cb529-8793-4ae3-afa1-f1bd9d5ae5f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:52.819471 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7cb529-8793-4ae3-afa1-f1bd9d5ae5f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:52.821266 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7cb529-8793-4ae3-afa1-f1bd9d5ae5f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:52.824816 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7cb529-8793-4ae3-afa1-f1bd9d5ae5f6 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:52.878742 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1363/2686] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:41:52 2022] POST /identity/v3/users => generated 288 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:41:52.885070 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d25a98c-3ac2-4ee6-bea7-219a7fd8bd19 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:52.901265 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d25a98c-3ac2-4ee6-bea7-219a7fd8bd19 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:52.907215 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06d4e9c-62dd-43a6-b793-f20441a388f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:52.907970 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d4e9c-62dd-43a6-b793-f20441a388f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:52.908286 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d4e9c-62dd-43a6-b793-f20441a388f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:52.908604 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d4e9c-62dd-43a6-b793-f20441a388f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:52.909671 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06d4e9c-62dd-43a6-b793-f20441a388f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:52.912300 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06d4e9c-62dd-43a6-b793-f20441a388f5 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:52.927241 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1324/2687] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:41:52 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:41:52.932135 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7cb529-8793-4ae3-afa1-f1bd9d5ae5f6 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:52.944361 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7cb529-8793-4ae3-afa1-f1bd9d5ae5f6 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:52.950417 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88076a1a-4693-4826-a3f3-2e705429e893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:52.951329 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88076a1a-4693-4826-a3f3-2e705429e893 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:52.951838 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88076a1a-4693-4826-a3f3-2e705429e893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:52.952297 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88076a1a-4693-4826-a3f3-2e705429e893 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:52.953545 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88076a1a-4693-4826-a3f3-2e705429e893 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:52.955761 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88076a1a-4693-4826-a3f3-2e705429e893 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:52.971935 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1364/2688] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:41:52 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:41:52.976850 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06d4e9c-62dd-43a6-b793-f20441a388f5 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:52.992354 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06d4e9c-62dd-43a6-b793-f20441a388f5 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:53.000701 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3939355c-dff4-4286-ad89-5a483b2fe9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:53.002307 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939355c-dff4-4286-ad89-5a483b2fe9bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:53.002767 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939355c-dff4-4286-ad89-5a483b2fe9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:53.003950 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939355c-dff4-4286-ad89-5a483b2fe9bc admin admin] PATH_INFO: `/v3/projects/feb99813da194ff7bac447a6f3413463/users/f8d41e4794584e2d85ee1714e229fad4/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:53.026708 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3939355c-dff4-4286-ad89-5a483b2fe9bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=feb99813da194ff7bac447a6f3413463, user_id=f8d41e4794584e2d85ee1714e229fad4, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:53.029689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3939355c-dff4-4286-ad89-5a483b2fe9bc admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:53.048165 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1325/2689] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:41:52 2022] PUT /identity/v3/projects/feb99813da194ff7bac447a6f3413463/users/f8d41e4794584e2d85ee1714e229fad4/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:41:53.064352 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead3d383-54a6-45e3-8c8e-0f420e42a6c9 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:53.064883 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead3d383-54a6-45e3-8c8e-0f420e42a6c9 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:53.065336 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead3d383-54a6-45e3-8c8e-0f420e42a6c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:53.119105 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-ead3d383-54a6-45e3-8c8e-0f420e42a6c9 None None] MFA Rules not processed for user `f8d41e4794584e2d85ee1714e229fad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:41:53.120493 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ead3d383-54a6-45e3-8c8e-0f420e42a6c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:53.194975 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:41:53.239315 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1365/2690] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:41:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:41:53.254261 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3939355c-dff4-4286-ad89-5a483b2fe9bc admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:53.256757 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3939355c-dff4-4286-ad89-5a483b2fe9bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:53.364980 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3939355c-dff4-4286-ad89-5a483b2fe9bc admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:53.371376 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae4d6b1-3b6d-448e-bbc6-52231981453f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:53.372369 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4d6b1-3b6d-448e-bbc6-52231981453f service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:53.372776 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4d6b1-3b6d-448e-bbc6-52231981453f service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:53.373177 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4d6b1-3b6d-448e-bbc6-52231981453f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:53.386659 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae4d6b1-3b6d-448e-bbc6-52231981453f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:53.388773 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae4d6b1-3b6d-448e-bbc6-52231981453f service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:53.404219 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1326/2691] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:41:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:41:59.473064 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead3d383-54a6-45e3-8c8e-0f420e42a6c9 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:59.486219 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ead3d383-54a6-45e3-8c8e-0f420e42a6c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:59.623660 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead3d383-54a6-45e3-8c8e-0f420e42a6c9 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:59.635018 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61c051-b057-47f6-94bb-2b9db550521f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:59.636594 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61c051-b057-47f6-94bb-2b9db550521f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:59.637363 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61c051-b057-47f6-94bb-2b9db550521f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:59.637828 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61c051-b057-47f6-94bb-2b9db550521f admin admin] PATH_INFO: `/v3/users/757b04583c444e2aa302d1078908a312` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:59.641834 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa61c051-b057-47f6-94bb-2b9db550521f admin admin] RBAC: Authorizing `identity:delete_user(user_id=757b04583c444e2aa302d1078908a312)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:59.644796 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa61c051-b057-47f6-94bb-2b9db550521f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:59.730694 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-fa61c051-b057-47f6-94bb-2b9db550521f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '757b04583c444e2aa302d1078908a312'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:59.745672 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-fa61c051-b057-47f6-94bb-2b9db550521f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '757b04583c444e2aa302d1078908a312'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:59.746485 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-fa61c051-b057-47f6-94bb-2b9db550521f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '757b04583c444e2aa302d1078908a312'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:59.747275 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-fa61c051-b057-47f6-94bb-2b9db550521f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '757b04583c444e2aa302d1078908a312'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:59.757610 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-fa61c051-b057-47f6-94bb-2b9db550521f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '757b04583c444e2aa302d1078908a312'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:41:59.773523 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1366/2692] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:41:59 2022] DELETE /identity/v3/users/757b04583c444e2aa302d1078908a312 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:41:59.780602 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa883821-0cf6-4062-8d77-ccaa0a0e7cf9 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:59.781355 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa883821-0cf6-4062-8d77-ccaa0a0e7cf9 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:59.781355 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa883821-0cf6-4062-8d77-ccaa0a0e7cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:59.802659 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-fa883821-0cf6-4062-8d77-ccaa0a0e7cf9 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:41:59.803990 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fa883821-0cf6-4062-8d77-ccaa0a0e7cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:59.895898 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:41:59.910184 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1327/2693] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:41:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:41:59.926939 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61c051-b057-47f6-94bb-2b9db550521f admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:59.929225 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-fa61c051-b057-47f6-94bb-2b9db550521f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:42:00.024374 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61c051-b057-47f6-94bb-2b9db550521f admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:00.028825 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483de481-1f05-491c-960e-1e7dc11b58a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:00.029708 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483de481-1f05-491c-960e-1e7dc11b58a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:00.030061 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483de481-1f05-491c-960e-1e7dc11b58a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:00.030409 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483de481-1f05-491c-960e-1e7dc11b58a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:00.044106 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483de481-1f05-491c-960e-1e7dc11b58a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:00.045835 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483de481-1f05-491c-960e-1e7dc11b58a0 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:00.059067 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1367/2694] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:41:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:42:00.467264 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa883821-0cf6-4062-8d77-ccaa0a0e7cf9 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:00.469974 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fa883821-0cf6-4062-8d77-ccaa0a0e7cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:42:00.490938 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa883821-0cf6-4062-8d77-ccaa0a0e7cf9 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:00.499291 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12371f7-44c5-4699-87c2-46173c206146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:00.500275 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12371f7-44c5-4699-87c2-46173c206146 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:00.500275 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12371f7-44c5-4699-87c2-46173c206146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:00.500275 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12371f7-44c5-4699-87c2-46173c206146 admin admin] PATH_INFO: `/v3/projects/63425ce0072941c9a3d59cd111fe691d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:00.503509 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12371f7-44c5-4699-87c2-46173c206146 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63425ce0072941c9a3d59cd111fe691d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:00.506631 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12371f7-44c5-4699-87c2-46173c206146 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:00.584856 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-f12371f7-44c5-4699-87c2-46173c206146 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63425ce0072941c9a3d59cd111fe691d', 'request_id': 'req-f12371f7-44c5-4699-87c2-46173c206146'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:42:00.598218 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-f12371f7-44c5-4699-87c2-46173c206146 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63425ce0072941c9a3d59cd111fe691d', 'request_id': 'req-f12371f7-44c5-4699-87c2-46173c206146'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:42:00.611441 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-f12371f7-44c5-4699-87c2-46173c206146 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63425ce0072941c9a3d59cd111fe691d', 'request_id': 'req-f12371f7-44c5-4699-87c2-46173c206146'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:42:00.633968 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-f12371f7-44c5-4699-87c2-46173c206146 admin admin] The token cache is being invalidate because project 63425ce0072941c9a3d59cd111fe691d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:42:00.634502 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-f12371f7-44c5-4699-87c2-46173c206146 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:42:00.635730 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-f12371f7-44c5-4699-87c2-46173c206146 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:42:00.638542 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1328/2695] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:42:00 2022] DELETE /identity/v3/projects/63425ce0072941c9a3d59cd111fe691d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:42:00.646148 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e272d-0335-42ed-9b64-c8525f8fdb2f None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:00.647608 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e272d-0335-42ed-9b64-c8525f8fdb2f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:00.648129 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e272d-0335-42ed-9b64-c8525f8fdb2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:00.670819 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-2e6e272d-0335-42ed-9b64-c8525f8fdb2f None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:42:00.672214 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-2e6e272d-0335-42ed-9b64-c8525f8fdb2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:42:00.677529 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:42:00.773947 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1368/2696] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:42:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:42:00.783424 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30460e7-49ea-4ada-aa33-5b8be9632593 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:00.784008 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30460e7-49ea-4ada-aa33-5b8be9632593 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:00.784113 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30460e7-49ea-4ada-aa33-5b8be9632593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:00.804893 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d30460e7-49ea-4ada-aa33-5b8be9632593 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:42:00.806373 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d30460e7-49ea-4ada-aa33-5b8be9632593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:42:00.815580 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1329/2697] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:42:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:42:00.820585 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e272d-0335-42ed-9b64-c8525f8fdb2f None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:00.831756 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e272d-0335-42ed-9b64-c8525f8fdb2f None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:00.838223 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544c0c8c-0878-4aee-ab7f-f050a28b3eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:00.839564 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c0c8c-0878-4aee-ab7f-f050a28b3eed admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:00.839968 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c0c8c-0878-4aee-ab7f-f050a28b3eed admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:00.840360 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c0c8c-0878-4aee-ab7f-f050a28b3eed admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:00.841505 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544c0c8c-0878-4aee-ab7f-f050a28b3eed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:42:00.842053 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544c0c8c-0878-4aee-ab7f-f050a28b3eed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:00.844558 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544c0c8c-0878-4aee-ab7f-f050a28b3eed admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:00.865698 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1369/2698] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:42:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:42:00.871038 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30460e7-49ea-4ada-aa33-5b8be9632593 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:00.883288 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30460e7-49ea-4ada-aa33-5b8be9632593 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:00.889276 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a0f61d-17fa-405c-be86-64fef5e54786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:00.890158 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0f61d-17fa-405c-be86-64fef5e54786 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:00.890655 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0f61d-17fa-405c-be86-64fef5e54786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:00.891003 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0f61d-17fa-405c-be86-64fef5e54786 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:00.891860 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a0f61d-17fa-405c-be86-64fef5e54786 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:00.894800 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a0f61d-17fa-405c-be86-64fef5e54786 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:00.930857 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1330/2699] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:42:00 2022] POST /identity/v3/projects => generated 391 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:42:00.937703 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544c0c8c-0878-4aee-ab7f-f050a28b3eed admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:00.954611 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544c0c8c-0878-4aee-ab7f-f050a28b3eed admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:01.040024 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deaed17-8aaf-4867-aef0-2eb53cfcf032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:01.040865 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deaed17-8aaf-4867-aef0-2eb53cfcf032 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:01.041102 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deaed17-8aaf-4867-aef0-2eb53cfcf032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:01.041704 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deaed17-8aaf-4867-aef0-2eb53cfcf032 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:01.042301 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deaed17-8aaf-4867-aef0-2eb53cfcf032 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:01.045199 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deaed17-8aaf-4867-aef0-2eb53cfcf032 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:01.077050 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1370/2700] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:42:00 2022] POST /identity/v3/users => generated 296 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:42:01.083063 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a0f61d-17fa-405c-be86-64fef5e54786 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:01.098863 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a0f61d-17fa-405c-be86-64fef5e54786 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:01.105901 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd596da-be38-438e-ad54-f5bb1840d7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:01.108690 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd596da-be38-438e-ad54-f5bb1840d7c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:01.108690 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd596da-be38-438e-ad54-f5bb1840d7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:01.108690 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd596da-be38-438e-ad54-f5bb1840d7c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:01.108690 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd596da-be38-438e-ad54-f5bb1840d7c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:01.109684 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd596da-be38-438e-ad54-f5bb1840d7c3 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:01.123372 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1331/2701] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:42:01 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:42:01.128764 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deaed17-8aaf-4867-aef0-2eb53cfcf032 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:01.143441 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deaed17-8aaf-4867-aef0-2eb53cfcf032 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:01.149396 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f579542-5faf-4ce4-87a8-046608169b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:01.150246 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f579542-5faf-4ce4-87a8-046608169b7c admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:01.150581 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f579542-5faf-4ce4-87a8-046608169b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:01.150901 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f579542-5faf-4ce4-87a8-046608169b7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:01.151728 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f579542-5faf-4ce4-87a8-046608169b7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:01.153876 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f579542-5faf-4ce4-87a8-046608169b7c admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:01.170260 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1371/2702] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:42:01 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:42:01.176031 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd596da-be38-438e-ad54-f5bb1840d7c3 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:01.188101 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd596da-be38-438e-ad54-f5bb1840d7c3 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:01.194074 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d78116f-7307-4baa-be64-33b415ddfb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:01.195221 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78116f-7307-4baa-be64-33b415ddfb78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:01.195617 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78116f-7307-4baa-be64-33b415ddfb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:01.196005 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78116f-7307-4baa-be64-33b415ddfb78 admin admin] PATH_INFO: `/v3/projects/4ff4a708c3344bd387cc0587c3a1359f/users/b790caefcb9a48d789545a513e870737/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:01.217432 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d78116f-7307-4baa-be64-33b415ddfb78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ff4a708c3344bd387cc0587c3a1359f, user_id=b790caefcb9a48d789545a513e870737, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:01.220716 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d78116f-7307-4baa-be64-33b415ddfb78 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:01.244431 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1332/2703] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:42:01 2022] PUT /identity/v3/projects/4ff4a708c3344bd387cc0587c3a1359f/users/b790caefcb9a48d789545a513e870737/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:42:01.260479 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b78b59-e930-40aa-8db8-b73a8ae6a319 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:01.260982 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b78b59-e930-40aa-8db8-b73a8ae6a319 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:01.261428 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b78b59-e930-40aa-8db8-b73a8ae6a319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:01.298667 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-92b78b59-e930-40aa-8db8-b73a8ae6a319 None None] MFA Rules not processed for user `b790caefcb9a48d789545a513e870737`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:42:01.299659 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-92b78b59-e930-40aa-8db8-b73a8ae6a319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:42:01.406056 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1372/2704] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:42:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:42:01.418625 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d78116f-7307-4baa-be64-33b415ddfb78 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:01.420576 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0d78116f-7307-4baa-be64-33b415ddfb78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:42:01.502137 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d78116f-7307-4baa-be64-33b415ddfb78 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:01.516159 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b958965c-dfa8-40b5-8151-54012a984335 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:01.516921 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958965c-dfa8-40b5-8151-54012a984335 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:01.517218 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958965c-dfa8-40b5-8151-54012a984335 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:01.517548 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958965c-dfa8-40b5-8151-54012a984335 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:01.534063 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b958965c-dfa8-40b5-8151-54012a984335 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:01.534794 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b958965c-dfa8-40b5-8151-54012a984335 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:01.549002 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1333/2705] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:42:01 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:42:11.402158 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=95,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:42:11.542316 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=80,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:42:29.084251 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b78b59-e930-40aa-8db8-b73a8ae6a319 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:29.086684 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-92b78b59-e930-40aa-8db8-b73a8ae6a319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:42:29.202236 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b78b59-e930-40aa-8db8-b73a8ae6a319 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:29.210101 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69992d30-b783-4ab6-aef2-b096a0760836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:29.211448 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69992d30-b783-4ab6-aef2-b096a0760836 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:29.211881 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69992d30-b783-4ab6-aef2-b096a0760836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:29.212292 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69992d30-b783-4ab6-aef2-b096a0760836 admin admin] PATH_INFO: `/v3/users/f8d41e4794584e2d85ee1714e229fad4` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:29.214272 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69992d30-b783-4ab6-aef2-b096a0760836 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8d41e4794584e2d85ee1714e229fad4)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:29.217998 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69992d30-b783-4ab6-aef2-b096a0760836 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:29.302664 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-69992d30-b783-4ab6-aef2-b096a0760836 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8d41e4794584e2d85ee1714e229fad4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:42:29.328396 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-69992d30-b783-4ab6-aef2-b096a0760836 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8d41e4794584e2d85ee1714e229fad4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:42:29.329665 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-69992d30-b783-4ab6-aef2-b096a0760836 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8d41e4794584e2d85ee1714e229fad4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:42:29.330680 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-69992d30-b783-4ab6-aef2-b096a0760836 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8d41e4794584e2d85ee1714e229fad4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:42:29.344193 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-69992d30-b783-4ab6-aef2-b096a0760836 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8d41e4794584e2d85ee1714e229fad4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:42:29.365797 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1373/2706] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:42:29 2022] DELETE /identity/v3/users/f8d41e4794584e2d85ee1714e229fad4 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:42:29.372529 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a5118-194c-45b8-a0f1-2639c554a3f6 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:29.373193 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a5118-194c-45b8-a0f1-2639c554a3f6 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:29.373403 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a5118-194c-45b8-a0f1-2639c554a3f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:29.401884 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3c8a5118-194c-45b8-a0f1-2639c554a3f6 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:42:29.403150 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3c8a5118-194c-45b8-a0f1-2639c554a3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:42:29.484177 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1334/2707] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:42:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:42:29.495238 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69992d30-b783-4ab6-aef2-b096a0760836 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:29.496866 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-69992d30-b783-4ab6-aef2-b096a0760836 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:42:29.561990 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69992d30-b783-4ab6-aef2-b096a0760836 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:29.567589 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c54bcf6-7db6-4ae3-a98b-74837d654e6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:29.568684 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54bcf6-7db6-4ae3-a98b-74837d654e6f service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:29.569152 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54bcf6-7db6-4ae3-a98b-74837d654e6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:29.569664 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54bcf6-7db6-4ae3-a98b-74837d654e6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:29.587626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c54bcf6-7db6-4ae3-a98b-74837d654e6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:29.590092 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c54bcf6-7db6-4ae3-a98b-74837d654e6f service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:29.608846 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1374/2708] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:42:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:42:30.037210 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8a5118-194c-45b8-a0f1-2639c554a3f6 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:30.039787 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3c8a5118-194c-45b8-a0f1-2639c554a3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:42:30.062387 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8a5118-194c-45b8-a0f1-2639c554a3f6 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:30.071291 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aefaa0-8904-4cba-8b58-d92ef1a0bed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:30.072854 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aefaa0-8904-4cba-8b58-d92ef1a0bed0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:30.073338 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aefaa0-8904-4cba-8b58-d92ef1a0bed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:30.073693 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aefaa0-8904-4cba-8b58-d92ef1a0bed0 admin admin] PATH_INFO: `/v3/projects/feb99813da194ff7bac447a6f3413463` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:30.075781 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aefaa0-8904-4cba-8b58-d92ef1a0bed0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=feb99813da194ff7bac447a6f3413463)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:30.079602 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aefaa0-8904-4cba-8b58-d92ef1a0bed0 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:30.145339 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-83aefaa0-8904-4cba-8b58-d92ef1a0bed0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'feb99813da194ff7bac447a6f3413463', 'request_id': 'req-83aefaa0-8904-4cba-8b58-d92ef1a0bed0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:42:30.156591 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-83aefaa0-8904-4cba-8b58-d92ef1a0bed0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'feb99813da194ff7bac447a6f3413463', 'request_id': 'req-83aefaa0-8904-4cba-8b58-d92ef1a0bed0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:42:30.164701 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-83aefaa0-8904-4cba-8b58-d92ef1a0bed0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'feb99813da194ff7bac447a6f3413463', 'request_id': 'req-83aefaa0-8904-4cba-8b58-d92ef1a0bed0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:42:30.182096 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-83aefaa0-8904-4cba-8b58-d92ef1a0bed0 admin admin] The token cache is being invalidate because project feb99813da194ff7bac447a6f3413463 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:42:30.182610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-83aefaa0-8904-4cba-8b58-d92ef1a0bed0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:42:30.183509 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-83aefaa0-8904-4cba-8b58-d92ef1a0bed0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:42:30.185689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1335/2709] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:42:30 2022] DELETE /identity/v3/projects/feb99813da194ff7bac447a6f3413463 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:42:30.193105 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f808042d-c2a6-4714-84f3-c0a3e0282be9 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:30.193944 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f808042d-c2a6-4714-84f3-c0a3e0282be9 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:30.194276 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f808042d-c2a6-4714-84f3-c0a3e0282be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:30.227791 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-f808042d-c2a6-4714-84f3-c0a3e0282be9 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:42:30.229512 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f808042d-c2a6-4714-84f3-c0a3e0282be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:42:30.260890 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:42:30.332649 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1375/2710] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:42:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:42:30.343138 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b2e66-08b8-4536-b117-735834bf6e6d None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:30.343600 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b2e66-08b8-4536-b117-735834bf6e6d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:30.343959 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b2e66-08b8-4536-b117-735834bf6e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:30.372074 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a56b2e66-08b8-4536-b117-735834bf6e6d None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:42:30.374071 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a56b2e66-08b8-4536-b117-735834bf6e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:42:30.388946 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1336/2711] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:42:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:42:30.395025 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f808042d-c2a6-4714-84f3-c0a3e0282be9 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:30.407218 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f808042d-c2a6-4714-84f3-c0a3e0282be9 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:30.412929 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18b4f31-fcbb-4c61-a332-ee48b6ada1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:30.414019 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b4f31-fcbb-4c61-a332-ee48b6ada1c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:30.414370 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b4f31-fcbb-4c61-a332-ee48b6ada1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:30.414706 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b4f31-fcbb-4c61-a332-ee48b6ada1c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:30.415537 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18b4f31-fcbb-4c61-a332-ee48b6ada1c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:42:30.415909 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18b4f31-fcbb-4c61-a332-ee48b6ada1c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:30.417765 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18b4f31-fcbb-4c61-a332-ee48b6ada1c1 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:30.433591 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1376/2712] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:42:30.440164 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56b2e66-08b8-4536-b117-735834bf6e6d None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:30.453440 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56b2e66-08b8-4536-b117-735834bf6e6d None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:30.459140 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd109e1-392e-4fa7-ad79-78b86eace1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:30.459831 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd109e1-392e-4fa7-ad79-78b86eace1bf admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:30.460126 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd109e1-392e-4fa7-ad79-78b86eace1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:30.460409 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd109e1-392e-4fa7-ad79-78b86eace1bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:30.461140 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd109e1-392e-4fa7-ad79-78b86eace1bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:30.463299 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd109e1-392e-4fa7-ad79-78b86eace1bf admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:30.504004 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1337/2713] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:42:30 2022] POST /identity/v3/projects => generated 395 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:42:30.509619 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18b4f31-fcbb-4c61-a332-ee48b6ada1c1 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:30.528731 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18b4f31-fcbb-4c61-a332-ee48b6ada1c1 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:30.623864 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cdc058-195d-4b45-ae6b-f9bf9690bb04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:30.625357 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cdc058-195d-4b45-ae6b-f9bf9690bb04 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:30.625899 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cdc058-195d-4b45-ae6b-f9bf9690bb04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:30.627063 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cdc058-195d-4b45-ae6b-f9bf9690bb04 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:30.627652 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cdc058-195d-4b45-ae6b-f9bf9690bb04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:30.631764 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cdc058-195d-4b45-ae6b-f9bf9690bb04 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:30.662183 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1377/2714] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:42:30 2022] POST /identity/v3/users => generated 298 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:42:30.667209 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd109e1-392e-4fa7-ad79-78b86eace1bf admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:30.679295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd109e1-392e-4fa7-ad79-78b86eace1bf admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:30.684490 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c26293d-bfa2-4298-90b7-8187cfcd3f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:30.685195 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26293d-bfa2-4298-90b7-8187cfcd3f2f admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:30.685483 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26293d-bfa2-4298-90b7-8187cfcd3f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:30.685812 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26293d-bfa2-4298-90b7-8187cfcd3f2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:30.686551 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c26293d-bfa2-4298-90b7-8187cfcd3f2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:30.688319 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c26293d-bfa2-4298-90b7-8187cfcd3f2f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:30.701067 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1338/2715] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:42:30.705793 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cdc058-195d-4b45-ae6b-f9bf9690bb04 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:30.717694 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cdc058-195d-4b45-ae6b-f9bf9690bb04 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:30.724224 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb58ce1-649d-42f4-b4fa-6c84106849d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:30.725027 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb58ce1-649d-42f4-b4fa-6c84106849d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:30.725779 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb58ce1-649d-42f4-b4fa-6c84106849d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:30.726264 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb58ce1-649d-42f4-b4fa-6c84106849d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:30.727085 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb58ce1-649d-42f4-b4fa-6c84106849d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:30.729304 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb58ce1-649d-42f4-b4fa-6c84106849d6 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:30.743801 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1378/2716] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:42:30.749774 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c26293d-bfa2-4298-90b7-8187cfcd3f2f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:30.761574 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c26293d-bfa2-4298-90b7-8187cfcd3f2f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:30.767195 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe6804f-5ef7-4f61-981f-c0c5726731b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:30.768584 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6804f-5ef7-4f61-981f-c0c5726731b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:30.768951 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6804f-5ef7-4f61-981f-c0c5726731b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:30.769397 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6804f-5ef7-4f61-981f-c0c5726731b8 admin admin] PATH_INFO: `/v3/projects/ae04cd64beb94a1eadf830214d470b71/users/0fc20e315e4f4fa1b1346d53cd4ac70f/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:30.787482 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe6804f-5ef7-4f61-981f-c0c5726731b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae04cd64beb94a1eadf830214d470b71, user_id=0fc20e315e4f4fa1b1346d53cd4ac70f, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:30.790336 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe6804f-5ef7-4f61-981f-c0c5726731b8 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:30.806924 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1339/2717] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:42:30 2022] PUT /identity/v3/projects/ae04cd64beb94a1eadf830214d470b71/users/0fc20e315e4f4fa1b1346d53cd4ac70f/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:42:30.822619 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8611a20-3898-4265-9858-1f89aa7cf52c None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:30.822933 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8611a20-3898-4265-9858-1f89aa7cf52c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:30.823206 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8611a20-3898-4265-9858-1f89aa7cf52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:30.855591 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-f8611a20-3898-4265-9858-1f89aa7cf52c None None] MFA Rules not processed for user `0fc20e315e4f4fa1b1346d53cd4ac70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:42:30.857228 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f8611a20-3898-4265-9858-1f89aa7cf52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:42:30.948333 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1379/2718] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:42:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:42:30.968412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe6804f-5ef7-4f61-981f-c0c5726731b8 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:30.971286 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-dbe6804f-5ef7-4f61-981f-c0c5726731b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:42:31.009324 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:42:31.054113 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe6804f-5ef7-4f61-981f-c0c5726731b8 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:31.058126 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076da9d1-c4f2-4791-879b-417ce35acdd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:31.058806 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076da9d1-c4f2-4791-879b-417ce35acdd7 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:31.059134 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076da9d1-c4f2-4791-879b-417ce35acdd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:31.059416 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076da9d1-c4f2-4791-879b-417ce35acdd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:31.070237 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076da9d1-c4f2-4791-879b-417ce35acdd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:31.073698 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076da9d1-c4f2-4791-879b-417ce35acdd7 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:31.084389 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1340/2719] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:42:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:42:31.461097 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8611a20-3898-4265-9858-1f89aa7cf52c None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:31.464013 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f8611a20-3898-4265-9858-1f89aa7cf52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:42:31.505946 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:42:31.582115 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8611a20-3898-4265-9858-1f89aa7cf52c None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:31.589417 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8c8b8-4f51-4ec8-ad9f-ef81b716e7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:31.591242 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8c8b8-4f51-4ec8-ad9f-ef81b716e7b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:31.591739 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8c8b8-4f51-4ec8-ad9f-ef81b716e7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:31.592250 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8c8b8-4f51-4ec8-ad9f-ef81b716e7b1 admin admin] PATH_INFO: `/v3/users/b790caefcb9a48d789545a513e870737` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:31.594899 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd8c8b8-4f51-4ec8-ad9f-ef81b716e7b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b790caefcb9a48d789545a513e870737)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:31.598998 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd8c8b8-4f51-4ec8-ad9f-ef81b716e7b1 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:31.686959 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-ecd8c8b8-4f51-4ec8-ad9f-ef81b716e7b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b790caefcb9a48d789545a513e870737'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:42:31.714713 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-ecd8c8b8-4f51-4ec8-ad9f-ef81b716e7b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b790caefcb9a48d789545a513e870737'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:42:31.715725 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-ecd8c8b8-4f51-4ec8-ad9f-ef81b716e7b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b790caefcb9a48d789545a513e870737'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:42:31.716464 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-ecd8c8b8-4f51-4ec8-ad9f-ef81b716e7b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b790caefcb9a48d789545a513e870737'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:42:31.728027 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-ecd8c8b8-4f51-4ec8-ad9f-ef81b716e7b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b790caefcb9a48d789545a513e870737'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:42:31.746459 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1380/2720] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:42:31 2022] DELETE /identity/v3/users/b790caefcb9a48d789545a513e870737 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:42:31.755336 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406dfd4-3e24-4dee-96bb-8ce7cf60e849 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:31.756210 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406dfd4-3e24-4dee-96bb-8ce7cf60e849 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:31.756384 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406dfd4-3e24-4dee-96bb-8ce7cf60e849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:31.792159 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3406dfd4-3e24-4dee-96bb-8ce7cf60e849 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:42:31.794537 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3406dfd4-3e24-4dee-96bb-8ce7cf60e849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:42:31.905074 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1341/2721] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:42:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:42:31.918201 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8c8b8-4f51-4ec8-ad9f-ef81b716e7b1 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:31.920038 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ecd8c8b8-4f51-4ec8-ad9f-ef81b716e7b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:42:31.993443 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8c8b8-4f51-4ec8-ad9f-ef81b716e7b1 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:31.998424 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a425ac-7122-4661-a831-b34f8127ce48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:31.998965 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a425ac-7122-4661-a831-b34f8127ce48 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:31.999253 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a425ac-7122-4661-a831-b34f8127ce48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:31.999534 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a425ac-7122-4661-a831-b34f8127ce48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:32.012602 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a425ac-7122-4661-a831-b34f8127ce48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:32.014138 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a425ac-7122-4661-a831-b34f8127ce48 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:32.026712 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1381/2722] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:42:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:42:32.404814 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3406dfd4-3e24-4dee-96bb-8ce7cf60e849 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:32.407452 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3406dfd4-3e24-4dee-96bb-8ce7cf60e849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:42:32.424813 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3406dfd4-3e24-4dee-96bb-8ce7cf60e849 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:32.432287 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed4354e-32b0-4998-97b3-2e1b8759c328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:32.433858 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4354e-32b0-4998-97b3-2e1b8759c328 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:32.434358 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4354e-32b0-4998-97b3-2e1b8759c328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:32.434854 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4354e-32b0-4998-97b3-2e1b8759c328 admin admin] PATH_INFO: `/v3/projects/4ff4a708c3344bd387cc0587c3a1359f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:32.437087 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed4354e-32b0-4998-97b3-2e1b8759c328 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ff4a708c3344bd387cc0587c3a1359f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:32.439244 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed4354e-32b0-4998-97b3-2e1b8759c328 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:32.496776 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8ed4354e-32b0-4998-97b3-2e1b8759c328 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ff4a708c3344bd387cc0587c3a1359f', 'request_id': 'req-8ed4354e-32b0-4998-97b3-2e1b8759c328'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:42:32.506393 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8ed4354e-32b0-4998-97b3-2e1b8759c328 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ff4a708c3344bd387cc0587c3a1359f', 'request_id': 'req-8ed4354e-32b0-4998-97b3-2e1b8759c328'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:42:32.512560 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8ed4354e-32b0-4998-97b3-2e1b8759c328 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ff4a708c3344bd387cc0587c3a1359f', 'request_id': 'req-8ed4354e-32b0-4998-97b3-2e1b8759c328'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:42:32.525424 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8ed4354e-32b0-4998-97b3-2e1b8759c328 admin admin] The token cache is being invalidate because project 4ff4a708c3344bd387cc0587c3a1359f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:42:32.525768 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8ed4354e-32b0-4998-97b3-2e1b8759c328 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:42:32.526388 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8ed4354e-32b0-4998-97b3-2e1b8759c328 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:42:32.528033 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1342/2723] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:42:32 2022] DELETE /identity/v3/projects/4ff4a708c3344bd387cc0587c3a1359f => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:42:32.533497 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a923048-3c42-4d2c-85cc-1b285a7f867f None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:32.534070 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a923048-3c42-4d2c-85cc-1b285a7f867f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:32.534180 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a923048-3c42-4d2c-85cc-1b285a7f867f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:32.554543 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-2a923048-3c42-4d2c-85cc-1b285a7f867f None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:42:32.555802 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-2a923048-3c42-4d2c-85cc-1b285a7f867f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:42:32.604546 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:42:32.643220 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1382/2724] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:42:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:42:32.652537 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993545e9-d2f8-4c2f-b422-b5761a997ae5 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:32.652872 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993545e9-d2f8-4c2f-b422-b5761a997ae5 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:32.653162 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993545e9-d2f8-4c2f-b422-b5761a997ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:32.674347 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-993545e9-d2f8-4c2f-b422-b5761a997ae5 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:42:32.675792 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-993545e9-d2f8-4c2f-b422-b5761a997ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:42:32.684385 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1343/2725] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:42:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:42:32.689324 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a923048-3c42-4d2c-85cc-1b285a7f867f None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:32.701298 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a923048-3c42-4d2c-85cc-1b285a7f867f None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:32.707620 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67778ff9-fffd-4b2a-b163-710413fcad06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:32.708304 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67778ff9-fffd-4b2a-b163-710413fcad06 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:32.708595 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67778ff9-fffd-4b2a-b163-710413fcad06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:32.708877 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67778ff9-fffd-4b2a-b163-710413fcad06 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:32.711526 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67778ff9-fffd-4b2a-b163-710413fcad06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:42:32.711526 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67778ff9-fffd-4b2a-b163-710413fcad06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:32.711948 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67778ff9-fffd-4b2a-b163-710413fcad06 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:32.732159 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1383/2726] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:42:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:42:32.738380 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993545e9-d2f8-4c2f-b422-b5761a997ae5 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:32.754993 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993545e9-d2f8-4c2f-b422-b5761a997ae5 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:32.760529 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d5972f-e9f3-42a1-b92a-457e78347743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:32.761820 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5972f-e9f3-42a1-b92a-457e78347743 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:32.762206 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5972f-e9f3-42a1-b92a-457e78347743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:32.762579 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5972f-e9f3-42a1-b92a-457e78347743 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:32.763537 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d5972f-e9f3-42a1-b92a-457e78347743 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:32.768049 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d5972f-e9f3-42a1-b92a-457e78347743 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:32.821744 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1344/2727] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:42:32 2022] POST /identity/v3/projects => generated 377 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:42:32.824543 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67778ff9-fffd-4b2a-b163-710413fcad06 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:32.839069 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67778ff9-fffd-4b2a-b163-710413fcad06 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:32.929789 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e8ed4e-d9d0-407b-8dfa-dc3ea6ac9f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:32.930618 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e8ed4e-d9d0-407b-8dfa-dc3ea6ac9f1c admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:32.930960 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e8ed4e-d9d0-407b-8dfa-dc3ea6ac9f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:32.931433 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e8ed4e-d9d0-407b-8dfa-dc3ea6ac9f1c admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:32.932559 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e8ed4e-d9d0-407b-8dfa-dc3ea6ac9f1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:32.935638 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e8ed4e-d9d0-407b-8dfa-dc3ea6ac9f1c admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:32.968297 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1384/2728] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:42:32 2022] POST /identity/v3/users => generated 289 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:42:32.976209 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d5972f-e9f3-42a1-b92a-457e78347743 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:32.988938 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d5972f-e9f3-42a1-b92a-457e78347743 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:32.995221 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17f08b6-4445-4683-9f25-e0743d926c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:32.996770 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f08b6-4445-4683-9f25-e0743d926c8f admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:32.996770 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f08b6-4445-4683-9f25-e0743d926c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:32.997120 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f08b6-4445-4683-9f25-e0743d926c8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:32.998664 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17f08b6-4445-4683-9f25-e0743d926c8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:32.999520 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17f08b6-4445-4683-9f25-e0743d926c8f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:33.015637 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1345/2729] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:42:32 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:42:33.020640 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e8ed4e-d9d0-407b-8dfa-dc3ea6ac9f1c admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:33.035021 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e8ed4e-d9d0-407b-8dfa-dc3ea6ac9f1c admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:33.040246 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ba9d9-39cd-4dda-b624-32552998716a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:33.040946 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ba9d9-39cd-4dda-b624-32552998716a admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:33.041288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ba9d9-39cd-4dda-b624-32552998716a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:33.041578 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ba9d9-39cd-4dda-b624-32552998716a admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:33.042368 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1ba9d9-39cd-4dda-b624-32552998716a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:33.044254 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1ba9d9-39cd-4dda-b624-32552998716a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:33.058754 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1385/2730] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:42:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:42:33.063375 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17f08b6-4445-4683-9f25-e0743d926c8f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:33.075670 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17f08b6-4445-4683-9f25-e0743d926c8f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:33.081994 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8821eb9-72ab-4e7c-ab7c-6d3e7c4e0fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:33.083975 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8821eb9-72ab-4e7c-ab7c-6d3e7c4e0fee admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:33.084305 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8821eb9-72ab-4e7c-ab7c-6d3e7c4e0fee admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:33.084638 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8821eb9-72ab-4e7c-ab7c-6d3e7c4e0fee admin admin] PATH_INFO: `/v3/projects/88c0ee0e3bda4c9b98c295af8718eff6/users/0628da68245a42e4af2e16e42babc15a/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:33.087906 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:42:33.109239 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8821eb9-72ab-4e7c-ab7c-6d3e7c4e0fee admin admin] RBAC: Authorizing `identity:create_grant(project_id=88c0ee0e3bda4c9b98c295af8718eff6, user_id=0628da68245a42e4af2e16e42babc15a, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:33.112237 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8821eb9-72ab-4e7c-ab7c-6d3e7c4e0fee admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:33.134836 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1346/2731] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:42:33 2022] PUT /identity/v3/projects/88c0ee0e3bda4c9b98c295af8718eff6/users/0628da68245a42e4af2e16e42babc15a/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:42:33.141920 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14adce35-39ba-4c40-8eb7-bef50e6b4fb0 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:33.142349 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14adce35-39ba-4c40-8eb7-bef50e6b4fb0 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:33.142725 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14adce35-39ba-4c40-8eb7-bef50e6b4fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:33.164458 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-14adce35-39ba-4c40-8eb7-bef50e6b4fb0 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:42:33.165739 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-14adce35-39ba-4c40-8eb7-bef50e6b4fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:42:33.254598 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1386/2732] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:42:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:42:33.266203 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8821eb9-72ab-4e7c-ab7c-6d3e7c4e0fee admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:33.268145 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c8821eb9-72ab-4e7c-ab7c-6d3e7c4e0fee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:42:33.338364 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8821eb9-72ab-4e7c-ab7c-6d3e7c4e0fee admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:33.342980 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3127aa23-f21a-42f8-a0dd-4bca5fc06cad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:33.343657 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3127aa23-f21a-42f8-a0dd-4bca5fc06cad service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:33.343945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3127aa23-f21a-42f8-a0dd-4bca5fc06cad service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:33.344225 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3127aa23-f21a-42f8-a0dd-4bca5fc06cad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:33.356062 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3127aa23-f21a-42f8-a0dd-4bca5fc06cad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:33.358055 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3127aa23-f21a-42f8-a0dd-4bca5fc06cad service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:33.369938 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1347/2733] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:42:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:42:38.729712 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddffaf3-cfd4-4b75-844d-c39b0821a69d None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:38.730572 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddffaf3-cfd4-4b75-844d-c39b0821a69d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:38.730572 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddffaf3-cfd4-4b75-844d-c39b0821a69d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:38.780822 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-3ddffaf3-cfd4-4b75-844d-c39b0821a69d None None] MFA Rules not processed for user `0628da68245a42e4af2e16e42babc15a`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:42:38.783289 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3ddffaf3-cfd4-4b75-844d-c39b0821a69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:42:38.852183 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:42:38.919795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1387/2734] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:42:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:42:38.928410 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3127aa23-f21a-42f8-a0dd-4bca5fc06cad service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:38.945975 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3127aa23-f21a-42f8-a0dd-4bca5fc06cad service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:38.952701 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490e79fc-1fd1-441a-97b3-4c52c6b68693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:38.953719 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490e79fc-1fd1-441a-97b3-4c52c6b68693 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:38.954217 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490e79fc-1fd1-441a-97b3-4c52c6b68693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:38.954547 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490e79fc-1fd1-441a-97b3-4c52c6b68693 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:38.955261 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490e79fc-1fd1-441a-97b3-4c52c6b68693 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:38.958657 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490e79fc-1fd1-441a-97b3-4c52c6b68693 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:39.015874 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1348/2735] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:42:38 2022] POST /identity/v3/projects => generated 379 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:42:39.025870 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddffaf3-cfd4-4b75-844d-c39b0821a69d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:39.045903 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddffaf3-cfd4-4b75-844d-c39b0821a69d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:39.133335 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9901f227-c625-47e3-9cba-f84ca8d755ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:39.134537 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901f227-c625-47e3-9cba-f84ca8d755ae admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:39.134998 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901f227-c625-47e3-9cba-f84ca8d755ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:39.135463 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901f227-c625-47e3-9cba-f84ca8d755ae admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:39.136356 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9901f227-c625-47e3-9cba-f84ca8d755ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:39.138896 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9901f227-c625-47e3-9cba-f84ca8d755ae admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:39.173148 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1388/2736] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:42:39 2022] POST /identity/v3/users => generated 296 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:42:39.178464 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490e79fc-1fd1-441a-97b3-4c52c6b68693 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:39.193434 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490e79fc-1fd1-441a-97b3-4c52c6b68693 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:39.199032 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254916a7-485e-4a4d-b5e1-66cd5039a5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:39.199736 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254916a7-485e-4a4d-b5e1-66cd5039a5ba admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:39.200029 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254916a7-485e-4a4d-b5e1-66cd5039a5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:39.200314 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254916a7-485e-4a4d-b5e1-66cd5039a5ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:39.201076 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254916a7-485e-4a4d-b5e1-66cd5039a5ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:39.202758 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254916a7-485e-4a4d-b5e1-66cd5039a5ba admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:39.218497 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1349/2737] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:42:39 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:42:39.224181 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9901f227-c625-47e3-9cba-f84ca8d755ae admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:39.235172 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9901f227-c625-47e3-9cba-f84ca8d755ae admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:39.240734 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a680919-3218-4615-af5c-956641031519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:39.243022 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a680919-3218-4615-af5c-956641031519 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:39.243022 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a680919-3218-4615-af5c-956641031519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:39.243022 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a680919-3218-4615-af5c-956641031519 admin admin] PATH_INFO: `/v3/projects/39e16f89051b42489c6d7775bef204a4/users/f620a64d26a541d788defef196a0eb91/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:39.269194 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a680919-3218-4615-af5c-956641031519 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39e16f89051b42489c6d7775bef204a4, user_id=f620a64d26a541d788defef196a0eb91, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:39.272840 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a680919-3218-4615-af5c-956641031519 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:39.299196 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1389/2738] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:42:39 2022] PUT /identity/v3/projects/39e16f89051b42489c6d7775bef204a4/users/f620a64d26a541d788defef196a0eb91/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:42:39.304788 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254916a7-485e-4a4d-b5e1-66cd5039a5ba admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:39.317078 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254916a7-485e-4a4d-b5e1-66cd5039a5ba admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:39.406482 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1f7d57-b35c-44df-a1f3-4ac1614c0834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:39.407348 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f7d57-b35c-44df-a1f3-4ac1614c0834 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:39.407643 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f7d57-b35c-44df-a1f3-4ac1614c0834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:39.407927 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f7d57-b35c-44df-a1f3-4ac1614c0834 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:39.408693 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1f7d57-b35c-44df-a1f3-4ac1614c0834 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:39.410433 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1f7d57-b35c-44df-a1f3-4ac1614c0834 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:39.427850 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1350/2739] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:42:39 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:42:39.433846 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a680919-3218-4615-af5c-956641031519 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:39.446894 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a680919-3218-4615-af5c-956641031519 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:39.452157 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f47de58-3244-4587-b047-445e04ec5045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:39.453082 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f47de58-3244-4587-b047-445e04ec5045 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:39.453365 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f47de58-3244-4587-b047-445e04ec5045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:39.453792 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f47de58-3244-4587-b047-445e04ec5045 admin admin] PATH_INFO: `/v3/projects/39e16f89051b42489c6d7775bef204a4/users/f620a64d26a541d788defef196a0eb91/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:39.456343 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f47de58-3244-4587-b047-445e04ec5045 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39e16f89051b42489c6d7775bef204a4, user_id=f620a64d26a541d788defef196a0eb91, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:39.459206 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f47de58-3244-4587-b047-445e04ec5045 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:39.474768 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1390/2740] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:42:39 2022] PUT /identity/v3/projects/39e16f89051b42489c6d7775bef204a4/users/f620a64d26a541d788defef196a0eb91/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:42:39.719940 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1f7d57-b35c-44df-a1f3-4ac1614c0834 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:39.724756 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7b1f7d57-b35c-44df-a1f3-4ac1614c0834 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:42:39.751579 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1f7d57-b35c-44df-a1f3-4ac1614c0834 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:39.844151 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:42:39.864413 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bcdd84-402f-41e2-b7e6-724a01f78d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:39.865606 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bcdd84-402f-41e2-b7e6-724a01f78d2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:39.865956 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bcdd84-402f-41e2-b7e6-724a01f78d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:39.866236 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bcdd84-402f-41e2-b7e6-724a01f78d2c admin admin] PATH_INFO: `/v3/users/0fc20e315e4f4fa1b1346d53cd4ac70f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:39.867630 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bcdd84-402f-41e2-b7e6-724a01f78d2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fc20e315e4f4fa1b1346d53cd4ac70f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:39.870267 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bcdd84-402f-41e2-b7e6-724a01f78d2c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:39.951719 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c3bcdd84-402f-41e2-b7e6-724a01f78d2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fc20e315e4f4fa1b1346d53cd4ac70f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:42:39.974102 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c3bcdd84-402f-41e2-b7e6-724a01f78d2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fc20e315e4f4fa1b1346d53cd4ac70f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:42:39.975376 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c3bcdd84-402f-41e2-b7e6-724a01f78d2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fc20e315e4f4fa1b1346d53cd4ac70f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:42:39.977797 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c3bcdd84-402f-41e2-b7e6-724a01f78d2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fc20e315e4f4fa1b1346d53cd4ac70f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:42:40.007714 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c3bcdd84-402f-41e2-b7e6-724a01f78d2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fc20e315e4f4fa1b1346d53cd4ac70f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:42:40.031945 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1351/2741] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:42:39 2022] DELETE /identity/v3/users/0fc20e315e4f4fa1b1346d53cd4ac70f => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:42:40.034857 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e428778f-66c9-4165-be52-1141b446a44e None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:40.037160 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e428778f-66c9-4165-be52-1141b446a44e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:40.037160 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e428778f-66c9-4165-be52-1141b446a44e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:40.073466 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-e428778f-66c9-4165-be52-1141b446a44e None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:42:40.075681 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e428778f-66c9-4165-be52-1141b446a44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:42:40.197962 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1391/2742] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:42:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:42:40.210637 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bcdd84-402f-41e2-b7e6-724a01f78d2c admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:40.212372 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c3bcdd84-402f-41e2-b7e6-724a01f78d2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:42:40.328573 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bcdd84-402f-41e2-b7e6-724a01f78d2c admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:40.338997 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560cc65f-5757-4131-a02f-babff594f468 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:40.341375 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560cc65f-5757-4131-a02f-babff594f468 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:40.341901 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560cc65f-5757-4131-a02f-babff594f468 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:40.342374 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560cc65f-5757-4131-a02f-babff594f468 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:40.360551 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560cc65f-5757-4131-a02f-babff594f468 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:40.363159 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560cc65f-5757-4131-a02f-babff594f468 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:40.385309 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1352/2743] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:42:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:42:40.904828 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e428778f-66c9-4165-be52-1141b446a44e None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:40.907109 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e428778f-66c9-4165-be52-1141b446a44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:42:40.937033 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e428778f-66c9-4165-be52-1141b446a44e None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:40.948450 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b69cbf9-5d38-406e-a53b-ed6067ae3785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:40.949410 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b69cbf9-5d38-406e-a53b-ed6067ae3785 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:40.949932 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b69cbf9-5d38-406e-a53b-ed6067ae3785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:40.950459 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b69cbf9-5d38-406e-a53b-ed6067ae3785 admin admin] PATH_INFO: `/v3/projects/ae04cd64beb94a1eadf830214d470b71` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:40.953057 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b69cbf9-5d38-406e-a53b-ed6067ae3785 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae04cd64beb94a1eadf830214d470b71)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:40.958209 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b69cbf9-5d38-406e-a53b-ed6067ae3785 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:41.013965 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:42:41.057183 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8b69cbf9-5d38-406e-a53b-ed6067ae3785 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae04cd64beb94a1eadf830214d470b71', 'request_id': 'req-8b69cbf9-5d38-406e-a53b-ed6067ae3785'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:42:41.069629 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8b69cbf9-5d38-406e-a53b-ed6067ae3785 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae04cd64beb94a1eadf830214d470b71', 'request_id': 'req-8b69cbf9-5d38-406e-a53b-ed6067ae3785'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:42:41.083878 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8b69cbf9-5d38-406e-a53b-ed6067ae3785 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae04cd64beb94a1eadf830214d470b71', 'request_id': 'req-8b69cbf9-5d38-406e-a53b-ed6067ae3785'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:42:41.106836 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8b69cbf9-5d38-406e-a53b-ed6067ae3785 admin admin] The token cache is being invalidate because project ae04cd64beb94a1eadf830214d470b71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:42:41.107379 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8b69cbf9-5d38-406e-a53b-ed6067ae3785 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:42:41.108685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8b69cbf9-5d38-406e-a53b-ed6067ae3785 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:42:41.111980 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1392/2744] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:42:40 2022] DELETE /identity/v3/projects/ae04cd64beb94a1eadf830214d470b71 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:42:41.118112 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269101e2-a283-438b-9748-1ba6e0f4c655 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:41.118934 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269101e2-a283-438b-9748-1ba6e0f4c655 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:41.119086 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269101e2-a283-438b-9748-1ba6e0f4c655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:41.143291 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-269101e2-a283-438b-9748-1ba6e0f4c655 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:42:41.146121 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-269101e2-a283-438b-9748-1ba6e0f4c655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:42:41.283240 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1353/2745] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:42:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:42:41.297079 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8956afb-d6bc-4c24-9f19-303e8423c04d None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:41.298045 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8956afb-d6bc-4c24-9f19-303e8423c04d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:41.298249 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8956afb-d6bc-4c24-9f19-303e8423c04d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:41.339055 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-f8956afb-d6bc-4c24-9f19-303e8423c04d None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:42:41.341744 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f8956afb-d6bc-4c24-9f19-303e8423c04d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:42:41.352723 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1393/2746] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:42:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:42:41.358826 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269101e2-a283-438b-9748-1ba6e0f4c655 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:41.403701 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269101e2-a283-438b-9748-1ba6e0f4c655 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:41.414531 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7419f1-00bc-4072-96a9-b775f9897f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:41.415704 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7419f1-00bc-4072-96a9-b775f9897f73 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:41.416222 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7419f1-00bc-4072-96a9-b775f9897f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:41.416797 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7419f1-00bc-4072-96a9-b775f9897f73 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:41.418416 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7419f1-00bc-4072-96a9-b775f9897f73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:42:41.419011 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7419f1-00bc-4072-96a9-b775f9897f73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:41.423768 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7419f1-00bc-4072-96a9-b775f9897f73 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:41.441250 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:42:41.453070 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1354/2747] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:42:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:42:41.459191 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8956afb-d6bc-4c24-9f19-303e8423c04d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:41.477886 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8956afb-d6bc-4c24-9f19-303e8423c04d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:41.486521 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fbaae4-bae4-4ed8-8b26-d016c1b5395c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:41.487521 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fbaae4-bae4-4ed8-8b26-d016c1b5395c admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:41.487988 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fbaae4-bae4-4ed8-8b26-d016c1b5395c admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:41.489184 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fbaae4-bae4-4ed8-8b26-d016c1b5395c admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:41.489943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fbaae4-bae4-4ed8-8b26-d016c1b5395c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:41.492937 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fbaae4-bae4-4ed8-8b26-d016c1b5395c admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:41.536358 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1394/2748] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:42:41 2022] POST /identity/v3/projects => generated 371 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:42:41.536567 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7419f1-00bc-4072-96a9-b775f9897f73 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:41.548733 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7419f1-00bc-4072-96a9-b775f9897f73 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:41.655913 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce285fb1-5110-4647-ae44-e82bdfd8ac35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:41.656905 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce285fb1-5110-4647-ae44-e82bdfd8ac35 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:41.657388 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce285fb1-5110-4647-ae44-e82bdfd8ac35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:41.657866 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce285fb1-5110-4647-ae44-e82bdfd8ac35 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:41.658722 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce285fb1-5110-4647-ae44-e82bdfd8ac35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:41.662861 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce285fb1-5110-4647-ae44-e82bdfd8ac35 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:41.698670 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1355/2749] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:42:41 2022] POST /identity/v3/users => generated 286 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:42:41.704301 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fbaae4-bae4-4ed8-8b26-d016c1b5395c admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:41.739363 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fbaae4-bae4-4ed8-8b26-d016c1b5395c admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:41.751984 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46362b9-4920-4391-9ec3-7e3371499816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:41.753617 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46362b9-4920-4391-9ec3-7e3371499816 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:41.754921 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46362b9-4920-4391-9ec3-7e3371499816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:41.755755 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46362b9-4920-4391-9ec3-7e3371499816 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:41.757264 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46362b9-4920-4391-9ec3-7e3371499816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:41.760208 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46362b9-4920-4391-9ec3-7e3371499816 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:41.778870 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1395/2750] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:42:41 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:42:41.780407 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce285fb1-5110-4647-ae44-e82bdfd8ac35 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:41.796568 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce285fb1-5110-4647-ae44-e82bdfd8ac35 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:41.802314 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1f569c-1c84-4b1c-9b34-a27a9e3a6909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:41.802898 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f569c-1c84-4b1c-9b34-a27a9e3a6909 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:41.803206 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f569c-1c84-4b1c-9b34-a27a9e3a6909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:41.803537 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f569c-1c84-4b1c-9b34-a27a9e3a6909 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:41.805085 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1f569c-1c84-4b1c-9b34-a27a9e3a6909 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:41.807849 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1f569c-1c84-4b1c-9b34-a27a9e3a6909 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:41.870126 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1356/2751] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:42:41 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:42:41.874580 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46362b9-4920-4391-9ec3-7e3371499816 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:41.900213 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46362b9-4920-4391-9ec3-7e3371499816 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:41.906222 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fabb562-32eb-4586-9fef-4f2dd0918709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:41.907311 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fabb562-32eb-4586-9fef-4f2dd0918709 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:41.907686 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fabb562-32eb-4586-9fef-4f2dd0918709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:41.908231 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fabb562-32eb-4586-9fef-4f2dd0918709 admin admin] PATH_INFO: `/v3/projects/990014bc19574d66ae5faae5c0751d5c/users/ea95ad1fabb74ba692aa9172128035eb/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:41.944521 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fabb562-32eb-4586-9fef-4f2dd0918709 admin admin] RBAC: Authorizing `identity:create_grant(project_id=990014bc19574d66ae5faae5c0751d5c, user_id=ea95ad1fabb74ba692aa9172128035eb, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:41.953555 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fabb562-32eb-4586-9fef-4f2dd0918709 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:41.993948 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1396/2752] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:42:41 2022] PUT /identity/v3/projects/990014bc19574d66ae5faae5c0751d5c/users/ea95ad1fabb74ba692aa9172128035eb/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:42:42.001162 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d55dc02-3b22-41a6-80c7-aa534fe8d735 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:42.001593 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d55dc02-3b22-41a6-80c7-aa534fe8d735 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:42.002016 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d55dc02-3b22-41a6-80c7-aa534fe8d735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:42.041805 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-7d55dc02-3b22-41a6-80c7-aa534fe8d735 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:42:42.043182 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7d55dc02-3b22-41a6-80c7-aa534fe8d735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:42:42.232754 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1357/2753] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:42:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:42:42.251090 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fabb562-32eb-4586-9fef-4f2dd0918709 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:42.253893 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7fabb562-32eb-4586-9fef-4f2dd0918709 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:42:42.367416 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fabb562-32eb-4586-9fef-4f2dd0918709 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:42.383309 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a5e681-8460-4452-a3e1-7f2f8a841a4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:42.385492 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a5e681-8460-4452-a3e1-7f2f8a841a4d service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:42.385999 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a5e681-8460-4452-a3e1-7f2f8a841a4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:42.386700 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a5e681-8460-4452-a3e1-7f2f8a841a4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:42.405588 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a5e681-8460-4452-a3e1-7f2f8a841a4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:42.408680 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a5e681-8460-4452-a3e1-7f2f8a841a4d service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:42.428211 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1397/2754] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:42:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:42:46.902152 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff3252-6ff2-40fb-8078-81aea07f9c66 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:46.903064 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff3252-6ff2-40fb-8078-81aea07f9c66 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:46.903064 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff3252-6ff2-40fb-8078-81aea07f9c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:46.937089 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-72ff3252-6ff2-40fb-8078-81aea07f9c66 None None] MFA Rules not processed for user `f620a64d26a541d788defef196a0eb91`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:42:46.938422 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-72ff3252-6ff2-40fb-8078-81aea07f9c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:42:47.065363 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1358/2755] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:42:46 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:42:47.086738 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a5e681-8460-4452-a3e1-7f2f8a841a4d service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:47.088891 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c0a5e681-8460-4452-a3e1-7f2f8a841a4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:42:47.173424 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:42:47.205050 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a5e681-8460-4452-a3e1-7f2f8a841a4d service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:47.214787 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343542eb-6a42-4b35-b226-ec2eb6930290 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:47.215639 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343542eb-6a42-4b35-b226-ec2eb6930290 service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:47.215947 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343542eb-6a42-4b35-b226-ec2eb6930290 service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:47.216238 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343542eb-6a42-4b35-b226-ec2eb6930290 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:47.220544 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-343542eb-6a42-4b35-b226-ec2eb6930290 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:42:47.370670 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343542eb-6a42-4b35-b226-ec2eb6930290 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:47.374279 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343542eb-6a42-4b35-b226-ec2eb6930290 service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:47.396096 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1398/2756] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:42:47 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:42:51.689529 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b30b8dc-fc49-4f04-9805-6b6e8712b2bc None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:51.691121 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b30b8dc-fc49-4f04-9805-6b6e8712b2bc None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:51.692196 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b30b8dc-fc49-4f04-9805-6b6e8712b2bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:51.698702 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:42:51.734085 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-9b30b8dc-fc49-4f04-9805-6b6e8712b2bc None None] MFA Rules not processed for user `ea95ad1fabb74ba692aa9172128035eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:42:51.738517 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9b30b8dc-fc49-4f04-9805-6b6e8712b2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:42:51.862628 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1359/2757] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:42:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:42:51.869200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343542eb-6a42-4b35-b226-ec2eb6930290 service nova] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:51.887344 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343542eb-6a42-4b35-b226-ec2eb6930290 service nova] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:51.892765 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf590299-ce15-44ad-aaf9-5b25e4d4e982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:51.893461 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf590299-ce15-44ad-aaf9-5b25e4d4e982 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:51.893795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf590299-ce15-44ad-aaf9-5b25e4d4e982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:51.894070 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf590299-ce15-44ad-aaf9-5b25e4d4e982 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:51.895794 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf590299-ce15-44ad-aaf9-5b25e4d4e982 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:51.897874 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf590299-ce15-44ad-aaf9-5b25e4d4e982 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:51.945961 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1399/2758] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:42:51 2022] POST /identity/v3/projects => generated 371 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:42:51.951768 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b30b8dc-fc49-4f04-9805-6b6e8712b2bc None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:51.967347 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b30b8dc-fc49-4f04-9805-6b6e8712b2bc None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:52.117975 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6af3fc-4575-455c-9a6f-ffd96443289e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:52.118990 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6af3fc-4575-455c-9a6f-ffd96443289e admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:52.119329 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6af3fc-4575-455c-9a6f-ffd96443289e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:52.119708 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6af3fc-4575-455c-9a6f-ffd96443289e admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:52.120509 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6af3fc-4575-455c-9a6f-ffd96443289e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:52.122958 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6af3fc-4575-455c-9a6f-ffd96443289e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:52.166324 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1360/2759] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:42:51 2022] POST /identity/v3/users => generated 292 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:42:52.171594 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf590299-ce15-44ad-aaf9-5b25e4d4e982 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:52.199391 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf590299-ce15-44ad-aaf9-5b25e4d4e982 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:52.223491 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e8182-59a2-45c7-b6f0-29d73b902f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:52.224751 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e8182-59a2-45c7-b6f0-29d73b902f9f admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:52.225163 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e8182-59a2-45c7-b6f0-29d73b902f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:52.229397 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e8182-59a2-45c7-b6f0-29d73b902f9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:52.229397 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6e8182-59a2-45c7-b6f0-29d73b902f9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:52.231500 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6e8182-59a2-45c7-b6f0-29d73b902f9f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:52.247621 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1400/2760] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:42:52 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:42:52.253499 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6af3fc-4575-455c-9a6f-ffd96443289e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:52.278699 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6af3fc-4575-455c-9a6f-ffd96443289e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:52.287915 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56342b71-cb9e-49dd-b83a-1b3fa2affb5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:52.289494 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56342b71-cb9e-49dd-b83a-1b3fa2affb5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:52.290089 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56342b71-cb9e-49dd-b83a-1b3fa2affb5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:52.292278 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56342b71-cb9e-49dd-b83a-1b3fa2affb5f admin admin] PATH_INFO: `/v3/projects/51a74b58d7534f4b9a2b613c9c1bf1ca/users/f1b5e6da52994804b1859c22cb739b06/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:52.327126 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56342b71-cb9e-49dd-b83a-1b3fa2affb5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=51a74b58d7534f4b9a2b613c9c1bf1ca, user_id=f1b5e6da52994804b1859c22cb739b06, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:52.333128 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56342b71-cb9e-49dd-b83a-1b3fa2affb5f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:52.356262 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1361/2761] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:42:52 2022] PUT /identity/v3/projects/51a74b58d7534f4b9a2b613c9c1bf1ca/users/f1b5e6da52994804b1859c22cb739b06/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:42:52.361046 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e8182-59a2-45c7-b6f0-29d73b902f9f admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:52.375500 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e8182-59a2-45c7-b6f0-29d73b902f9f admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:52.484509 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272f80d0-8de9-4947-8529-c478675d91a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:52.484509 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f80d0-8de9-4947-8529-c478675d91a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:52.484509 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f80d0-8de9-4947-8529-c478675d91a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:52.486460 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f80d0-8de9-4947-8529-c478675d91a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:52.486460 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272f80d0-8de9-4947-8529-c478675d91a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:52.490394 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272f80d0-8de9-4947-8529-c478675d91a6 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:52.512526 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1401/2762] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:42:52 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:42:52.516964 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56342b71-cb9e-49dd-b83a-1b3fa2affb5f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:52.546646 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56342b71-cb9e-49dd-b83a-1b3fa2affb5f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:52.555136 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68c343-6d4c-40f6-b880-86229a17f3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:52.556674 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68c343-6d4c-40f6-b880-86229a17f3e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:52.557238 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68c343-6d4c-40f6-b880-86229a17f3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:52.557775 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68c343-6d4c-40f6-b880-86229a17f3e2 admin admin] PATH_INFO: `/v3/projects/51a74b58d7534f4b9a2b613c9c1bf1ca/users/f1b5e6da52994804b1859c22cb739b06/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:52.560588 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b68c343-6d4c-40f6-b880-86229a17f3e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51a74b58d7534f4b9a2b613c9c1bf1ca, user_id=f1b5e6da52994804b1859c22cb739b06, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:52.563471 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b68c343-6d4c-40f6-b880-86229a17f3e2 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:52.586895 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1362/2763] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:42:52 2022] PUT /identity/v3/projects/51a74b58d7534f4b9a2b613c9c1bf1ca/users/f1b5e6da52994804b1859c22cb739b06/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:42:59.320964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb0886e-62d0-46be-a7f0-acac29dffae0 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:59.324472 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb0886e-62d0-46be-a7f0-acac29dffae0 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:59.326431 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb0886e-62d0-46be-a7f0-acac29dffae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:59.398534 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-0bb0886e-62d0-46be-a7f0-acac29dffae0 None None] MFA Rules not processed for user `f1b5e6da52994804b1859c22cb739b06`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:42:59.401102 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-0bb0886e-62d0-46be-a7f0-acac29dffae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:42:59.486096 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:42:59.567359 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1402/2764] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:42:59 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:42:59.568131 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68c343-6d4c-40f6-b880-86229a17f3e2 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:42:59.571104 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0b68c343-6d4c-40f6-b880-86229a17f3e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:42:59.650506 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:42:59.725194 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68c343-6d4c-40f6-b880-86229a17f3e2 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:42:59.736863 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01bf621-485c-4581-8bdd-820df4d86ad7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:42:59.738069 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01bf621-485c-4581-8bdd-820df4d86ad7 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:42:59.738596 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01bf621-485c-4581-8bdd-820df4d86ad7 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:42:59.739083 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01bf621-485c-4581-8bdd-820df4d86ad7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:42:59.853253 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01bf621-485c-4581-8bdd-820df4d86ad7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:42:59.855428 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01bf621-485c-4581-8bdd-820df4d86ad7 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:42:59.870076 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1363/2765] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:42:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:43:01.486187 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d69a6-c7c9-4439-8b9d-5b649d36d628 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:43:01.486804 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d69a6-c7c9-4439-8b9d-5b649d36d628 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:43:01.486804 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d69a6-c7c9-4439-8b9d-5b649d36d628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:43:01.510239 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-454d69a6-c7c9-4439-8b9d-5b649d36d628 None None] MFA Rules not processed for user `ea95ad1fabb74ba692aa9172128035eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:43:01.511617 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-454d69a6-c7c9-4439-8b9d-5b649d36d628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:43:01.523617 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1403/2766] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:43:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:43:03.672971 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01bf621-485c-4581-8bdd-820df4d86ad7 service nova] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:43:03.674912 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d01bf621-485c-4581-8bdd-820df4d86ad7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:43:03.839336 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01bf621-485c-4581-8bdd-820df4d86ad7 service nova] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:43:03.846907 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39026fb-4dd5-44a0-84db-c29429bec897 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef1d49b36cdc4665b82487be97e9dcd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:43:03.848013 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39026fb-4dd5-44a0-84db-c29429bec897 service placement] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:43:03.848610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39026fb-4dd5-44a0-84db-c29429bec897 service placement] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:43:03.848820 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39026fb-4dd5-44a0-84db-c29429bec897 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:43:03.857906 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d39026fb-4dd5-44a0-84db-c29429bec897 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:43:03.878347 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39026fb-4dd5-44a0-84db-c29429bec897 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:43:03.881085 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39026fb-4dd5-44a0-84db-c29429bec897 service placement] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:43:03.898502 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1364/2767] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:43:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:43:09.808346 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d69a6-c7c9-4439-8b9d-5b649d36d628 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:43:09.829097 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d69a6-c7c9-4439-8b9d-5b649d36d628 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:43:09.837303 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114dbad-2846-4dd2-bc69-327021c0fc9e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:43:09.845064 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114dbad-2846-4dd2-bc69-327021c0fc9e service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:43:09.845064 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114dbad-2846-4dd2-bc69-327021c0fc9e service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:43:09.845064 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114dbad-2846-4dd2-bc69-327021c0fc9e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:43:09.845064 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b114dbad-2846-4dd2-bc69-327021c0fc9e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:43:09.941465 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b114dbad-2846-4dd2-bc69-327021c0fc9e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:43:09.944702 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b114dbad-2846-4dd2-bc69-327021c0fc9e service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:43:09.960361 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1404/2768] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:43:09 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:43:13.891286 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=77 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:43:14.418800 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c6d9d-a7a7-46dc-96ec-e144c1667a7d None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:43:14.419707 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c6d9d-a7a7-46dc-96ec-e144c1667a7d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:43:14.420186 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c6d9d-a7a7-46dc-96ec-e144c1667a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:43:14.450833 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-e54c6d9d-a7a7-46dc-96ec-e144c1667a7d None None] MFA Rules not processed for user `0628da68245a42e4af2e16e42babc15a`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:43:14.452590 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e54c6d9d-a7a7-46dc-96ec-e144c1667a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:43:14.553785 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1365/2769] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:43:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:43:15.005375 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114dbad-2846-4dd2-bc69-327021c0fc9e service nova] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:43:15.090304 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114dbad-2846-4dd2-bc69-327021c0fc9e service nova] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:43:15.097002 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b39a8-3072-4ad4-8c71-a08261907358 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:43:15.098278 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b39a8-3072-4ad4-8c71-a08261907358 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:43:15.099320 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b39a8-3072-4ad4-8c71-a08261907358 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:43:15.099320 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b39a8-3072-4ad4-8c71-a08261907358 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:43:15.102616 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b36b39a8-3072-4ad4-8c71-a08261907358 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:43:15.121915 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36b39a8-3072-4ad4-8c71-a08261907358 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:43:15.124217 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36b39a8-3072-4ad4-8c71-a08261907358 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:43:15.140316 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1405/2770] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:43:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:43:15.414380 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45916e-46bb-430f-8a98-45880f508838 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:43:15.415300 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45916e-46bb-430f-8a98-45880f508838 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:43:15.415300 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45916e-46bb-430f-8a98-45880f508838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:43:15.443808 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-ec45916e-46bb-430f-8a98-45880f508838 None None] MFA Rules not processed for user `93785710c10749f9aa3d29b43965909c`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:43:15.446256 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ec45916e-46bb-430f-8a98-45880f508838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:43:15.456867 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1366/2771] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:43:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:43:15.935405 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b39a8-3072-4ad4-8c71-a08261907358 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:43:15.949353 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b39a8-3072-4ad4-8c71-a08261907358 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:43:15.955721 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3207c72-c8f7-469a-8700-12564ae03d23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:43:15.956982 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3207c72-c8f7-469a-8700-12564ae03d23 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:43:15.957535 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3207c72-c8f7-469a-8700-12564ae03d23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:43:15.957988 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3207c72-c8f7-469a-8700-12564ae03d23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:43:15.962887 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d3207c72-c8f7-469a-8700-12564ae03d23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:43:15.975176 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3207c72-c8f7-469a-8700-12564ae03d23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:43:15.976656 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3207c72-c8f7-469a-8700-12564ae03d23 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:43:15.993166 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1406/2772] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:43:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:43:19.907114 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec45916e-46bb-430f-8a98-45880f508838 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:43:19.929873 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec45916e-46bb-430f-8a98-45880f508838 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:43:19.949373 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1275a16-62d7-4e82-9c76-f220efd250e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:43:19.951491 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1275a16-62d7-4e82-9c76-f220efd250e8 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:43:19.952091 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1275a16-62d7-4e82-9c76-f220efd250e8 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:43:19.952607 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1275a16-62d7-4e82-9c76-f220efd250e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:43:19.955740 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f1275a16-62d7-4e82-9c76-f220efd250e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:43:19.988891 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1275a16-62d7-4e82-9c76-f220efd250e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:43:19.992350 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1275a16-62d7-4e82-9c76-f220efd250e8 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:43:20.018774 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1367/2773] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:43:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:43:25.982874 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=74 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:43:30.005737 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=34 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:43:37.914053 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3207c72-c8f7-469a-8700-12564ae03d23 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:43:37.915734 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d3207c72-c8f7-469a-8700-12564ae03d23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:43:37.928579 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3207c72-c8f7-469a-8700-12564ae03d23 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:43:37.933451 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c71bcb-8a97-466a-8b2a-690f902c98aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:43:37.934319 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c71bcb-8a97-466a-8b2a-690f902c98aa service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:43:37.934602 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c71bcb-8a97-466a-8b2a-690f902c98aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:43:37.934878 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c71bcb-8a97-466a-8b2a-690f902c98aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:43:38.012337 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c71bcb-8a97-466a-8b2a-690f902c98aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:43:38.014092 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c71bcb-8a97-466a-8b2a-690f902c98aa service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:43:38.026021 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1407/2774] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:43:37 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:43:45.331013 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1275a16-62d7-4e82-9c76-f220efd250e8 service nova] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:43:45.348453 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1275a16-62d7-4e82-9c76-f220efd250e8 service nova] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:43:45.354017 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664c9fe1-953c-47f4-906d-ce0dd121dec6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:43:45.354691 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664c9fe1-953c-47f4-906d-ce0dd121dec6 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:43:45.354976 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664c9fe1-953c-47f4-906d-ce0dd121dec6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:43:45.355321 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664c9fe1-953c-47f4-906d-ce0dd121dec6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:43:45.368695 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664c9fe1-953c-47f4-906d-ce0dd121dec6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:43:45.370396 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664c9fe1-953c-47f4-906d-ce0dd121dec6 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:43:45.383460 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1368/2775] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:43:45 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:43:48.022171 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=30 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:43:55.377506 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=6 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:44:49.662062 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c71bcb-8a97-466a-8b2a-690f902c98aa service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:44:49.663625 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b6c71bcb-8a97-466a-8b2a-690f902c98aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:44:49.799952 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c71bcb-8a97-466a-8b2a-690f902c98aa service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:44:49.804524 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0181137-cb8c-4268-895d-35e5b056bd1f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7da4bf6f3be47b4aa25c6240cb4be03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:44:49.805248 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0181137-cb8c-4268-895d-35e5b056bd1f service glance] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:44:49.805578 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0181137-cb8c-4268-895d-35e5b056bd1f service glance] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:44:49.805954 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0181137-cb8c-4268-895d-35e5b056bd1f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:44:49.808017 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a0181137-cb8c-4268-895d-35e5b056bd1f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:44:49.829840 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0181137-cb8c-4268-895d-35e5b056bd1f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:44:49.831286 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0181137-cb8c-4268-895d-35e5b056bd1f service glance] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:44:49.846355 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1408/2776] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:44:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:44:55.018562 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664c9fe1-953c-47f4-906d-ce0dd121dec6 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:44:55.028348 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664c9fe1-953c-47f4-906d-ce0dd121dec6 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:44:55.032784 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e768e044-ea5b-4e19-8597-a54b86b22cda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:44:55.033475 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e768e044-ea5b-4e19-8597-a54b86b22cda service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:44:55.033796 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e768e044-ea5b-4e19-8597-a54b86b22cda service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:44:55.034083 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e768e044-ea5b-4e19-8597-a54b86b22cda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:44:55.035989 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e768e044-ea5b-4e19-8597-a54b86b22cda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:44:55.051205 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e768e044-ea5b-4e19-8597-a54b86b22cda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:44:55.052776 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e768e044-ea5b-4e19-8597-a54b86b22cda service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:44:55.064994 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1369/2777] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:44:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:44:59.838688 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=35 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:45:02.305763 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0181137-cb8c-4268-895d-35e5b056bd1f service glance] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:45:02.318182 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0181137-cb8c-4268-895d-35e5b056bd1f service glance] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:45:02.323609 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5bf584-e6f1-424c-9f56-45bf91057932 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:45:02.324889 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5bf584-e6f1-424c-9f56-45bf91057932 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:45:02.325596 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5bf584-e6f1-424c-9f56-45bf91057932 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:45:02.326035 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5bf584-e6f1-424c-9f56-45bf91057932 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:45:02.328733 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-be5bf584-e6f1-424c-9f56-45bf91057932 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:45:02.346629 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5bf584-e6f1-424c-9f56-45bf91057932 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:45:02.349947 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5bf584-e6f1-424c-9f56-45bf91057932 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:45:02.365251 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1409/2778] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:45:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:45:05.059468 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=6 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:45:07.520831 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec197daa-238d-42ed-831d-a51f7c88f748 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:45:07.521936 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec197daa-238d-42ed-831d-a51f7c88f748 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:45:07.521936 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec197daa-238d-42ed-831d-a51f7c88f748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:45:07.588633 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-ec197daa-238d-42ed-831d-a51f7c88f748 None None] MFA Rules not processed for user `09c9fa7dccb8491c9ad3b43967345ea2`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:45:07.590042 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ec197daa-238d-42ed-831d-a51f7c88f748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:45:07.668338 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1370/2779] 10.222.0.87 () {58 vars in 1009 bytes} [Tue Jun 21 18:45:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:45:07.730517 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5bf584-e6f1-424c-9f56-45bf91057932 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:45:07.732018 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-be5bf584-e6f1-424c-9f56-45bf91057932 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:45:07.865265 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5bf584-e6f1-424c-9f56-45bf91057932 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:45:07.871391 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5dfceb-2399-4a97-99ff-927fd3eef879 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '515b3571045e4fe29565307c8ecc10b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:45:07.872059 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5dfceb-2399-4a97-99ff-927fd3eef879 service swift] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:45:07.872346 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5dfceb-2399-4a97-99ff-927fd3eef879 service swift] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:45:07.872626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5dfceb-2399-4a97-99ff-927fd3eef879 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:45:07.882828 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5dfceb-2399-4a97-99ff-927fd3eef879 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:45:07.884669 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5dfceb-2399-4a97-99ff-927fd3eef879 service swift] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:45:07.895993 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1410/2780] 10.222.0.87 () {60 vars in 1384 bytes} [Tue Jun 21 18:45:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 21 18:45:17.664200 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=40 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:45:17.891587 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=47 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:45:25.991249 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f510c47-fd2d-401f-b5c5-ef6e505446e5 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:45:25.991922 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f510c47-fd2d-401f-b5c5-ef6e505446e5 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:45:25.991922 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f510c47-fd2d-401f-b5c5-ef6e505446e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:45:26.012993 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-8f510c47-fd2d-401f-b5c5-ef6e505446e5 None None] MFA Rules not processed for user `09c9fa7dccb8491c9ad3b43967345ea2`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:45:26.014286 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8f510c47-fd2d-401f-b5c5-ef6e505446e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:45:26.025218 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1371/2781] 10.222.0.87 () {58 vars in 1009 bytes} [Tue Jun 21 18:45:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:45:26.040511 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5dfceb-2399-4a97-99ff-927fd3eef879 service swift] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:45:26.052609 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5dfceb-2399-4a97-99ff-927fd3eef879 service swift] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:45:26.058262 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a65a96-4a84-4464-b914-6e15a599ac69 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '515b3571045e4fe29565307c8ecc10b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:45:26.058931 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a65a96-4a84-4464-b914-6e15a599ac69 service swift] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:45:26.059215 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a65a96-4a84-4464-b914-6e15a599ac69 service swift] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:45:26.059491 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a65a96-4a84-4464-b914-6e15a599ac69 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:45:26.069512 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a65a96-4a84-4464-b914-6e15a599ac69 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:45:26.071337 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a65a96-4a84-4464-b914-6e15a599ac69 service swift] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:45:26.083066 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1411/2782] 10.222.0.87 () {60 vars in 1384 bytes} [Tue Jun 21 18:45:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 18:45:36.006353 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:45:36.078756 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=6 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:45:45.325100 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bea154-b315-45b2-bc2f-78e46fc66bbc None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:45:45.326229 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bea154-b315-45b2-bc2f-78e46fc66bbc None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:45:45.326229 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bea154-b315-45b2-bc2f-78e46fc66bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:45:45.355324 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-20bea154-b315-45b2-bc2f-78e46fc66bbc None None] MFA Rules not processed for user `09c9fa7dccb8491c9ad3b43967345ea2`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:45:45.357385 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-20bea154-b315-45b2-bc2f-78e46fc66bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:45:45.371060 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1372/2783] 10.222.0.87 () {58 vars in 967 bytes} [Tue Jun 21 18:45:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:45:45.385094 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a65a96-4a84-4464-b914-6e15a599ac69 service swift] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:45:45.396884 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a65a96-4a84-4464-b914-6e15a599ac69 service swift] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:45:45.403616 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670411dd-bf05-4d52-9783-7df019de56e1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '515b3571045e4fe29565307c8ecc10b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:45:45.404292 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670411dd-bf05-4d52-9783-7df019de56e1 service swift] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:45:45.404574 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670411dd-bf05-4d52-9783-7df019de56e1 service swift] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:45:45.404828 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670411dd-bf05-4d52-9783-7df019de56e1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:45:45.415789 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670411dd-bf05-4d52-9783-7df019de56e1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:45:45.417437 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670411dd-bf05-4d52-9783-7df019de56e1 service swift] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:45:45.428298 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1412/2784] 10.222.0.87 () {60 vars in 1384 bytes} [Tue Jun 21 18:45:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 18:45:55.345738 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:45:55.423272 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=6 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:46:03.362082 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a69451-5501-4817-91ed-041431b91a69 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:46:03.362941 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a69451-5501-4817-91ed-041431b91a69 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:46:03.363430 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a69451-5501-4817-91ed-041431b91a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:46:03.394912 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-74a69451-5501-4817-91ed-041431b91a69 None None] MFA Rules not processed for user `0628da68245a42e4af2e16e42babc15a`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:46:03.396358 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-74a69451-5501-4817-91ed-041431b91a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:46:03.404629 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1373/2785] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:46:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:46:04.387743 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670411dd-bf05-4d52-9783-7df019de56e1 service swift] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:46:04.389930 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-670411dd-bf05-4d52-9783-7df019de56e1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:46:04.406302 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670411dd-bf05-4d52-9783-7df019de56e1 service swift] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:46:04.412772 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bdff27-769e-4866-988b-0b88b524eb79 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef1d49b36cdc4665b82487be97e9dcd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:46:04.413436 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bdff27-769e-4866-988b-0b88b524eb79 service placement] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:46:04.413773 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bdff27-769e-4866-988b-0b88b524eb79 service placement] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:46:04.414058 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bdff27-769e-4866-988b-0b88b524eb79 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:46:04.416903 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f1bdff27-769e-4866-988b-0b88b524eb79 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:46:04.431604 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bdff27-769e-4866-988b-0b88b524eb79 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:46:04.433295 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bdff27-769e-4866-988b-0b88b524eb79 service placement] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:46:04.445281 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1413/2786] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:46:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:46:13.387539 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:46:14.439977 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=6 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:46:29.417394 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a69451-5501-4817-91ed-041431b91a69 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:46:29.433827 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a69451-5501-4817-91ed-041431b91a69 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:46:29.438262 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc021f8-061f-4737-a4aa-4d2924bcc14a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:46:29.438938 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc021f8-061f-4737-a4aa-4d2924bcc14a service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:46:29.439224 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc021f8-061f-4737-a4aa-4d2924bcc14a service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:46:29.439502 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc021f8-061f-4737-a4aa-4d2924bcc14a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:46:29.441528 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5dc021f8-061f-4737-a4aa-4d2924bcc14a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:46:29.454747 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc021f8-061f-4737-a4aa-4d2924bcc14a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:46:29.456546 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc021f8-061f-4737-a4aa-4d2924bcc14a service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:46:29.468699 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1374/2787] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:46:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:46:29.601915 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bdff27-769e-4866-988b-0b88b524eb79 service placement] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:46:29.613876 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bdff27-769e-4866-988b-0b88b524eb79 service placement] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:46:29.618730 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35cd291-7338-4794-8add-6119dae3261e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:46:29.619580 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35cd291-7338-4794-8add-6119dae3261e service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:46:29.620039 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35cd291-7338-4794-8add-6119dae3261e service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:46:29.620421 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35cd291-7338-4794-8add-6119dae3261e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:46:29.622710 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a35cd291-7338-4794-8add-6119dae3261e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:46:29.646734 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35cd291-7338-4794-8add-6119dae3261e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:46:29.648747 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35cd291-7338-4794-8add-6119dae3261e service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:46:29.663156 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1414/2788] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:46:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:46:31.344665 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc021f8-061f-4737-a4aa-4d2924bcc14a service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:46:31.362361 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc021f8-061f-4737-a4aa-4d2924bcc14a service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:46:31.369693 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d310b2d-a13d-4ee0-9f0d-7649aeaf4f72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:46:31.370379 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d310b2d-a13d-4ee0-9f0d-7649aeaf4f72 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:46:31.370669 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d310b2d-a13d-4ee0-9f0d-7649aeaf4f72 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:46:31.370951 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d310b2d-a13d-4ee0-9f0d-7649aeaf4f72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:46:31.373197 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4d310b2d-a13d-4ee0-9f0d-7649aeaf4f72 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:46:31.394442 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d310b2d-a13d-4ee0-9f0d-7649aeaf4f72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:46:31.397438 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d310b2d-a13d-4ee0-9f0d-7649aeaf4f72 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:46:31.421883 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1375/2789] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:46:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:46:39.656392 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=6 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:46:41.407488 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=12 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:46:45.616811 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35cd291-7338-4794-8add-6119dae3261e service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:46:45.632735 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35cd291-7338-4794-8add-6119dae3261e service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:46:45.639590 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b0b9a3-4465-4fb6-ab2d-026668a3049f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:46:45.640284 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b0b9a3-4465-4fb6-ab2d-026668a3049f service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:46:45.640577 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b0b9a3-4465-4fb6-ab2d-026668a3049f service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:46:45.641069 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b0b9a3-4465-4fb6-ab2d-026668a3049f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:46:45.645050 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c7b0b9a3-4465-4fb6-ab2d-026668a3049f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:46:45.659840 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b0b9a3-4465-4fb6-ab2d-026668a3049f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:46:45.662114 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b0b9a3-4465-4fb6-ab2d-026668a3049f service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:46:45.683906 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1415/2790] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:46:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:46:50.757428 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d310b2d-a13d-4ee0-9f0d-7649aeaf4f72 service nova] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:46:50.776246 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d310b2d-a13d-4ee0-9f0d-7649aeaf4f72 service nova] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:46:50.783346 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f059a408-6e5e-4308-a4a0-905711da3160 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:46:50.784469 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f059a408-6e5e-4308-a4a0-905711da3160 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:46:50.784964 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f059a408-6e5e-4308-a4a0-905711da3160 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:46:50.785440 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f059a408-6e5e-4308-a4a0-905711da3160 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:46:50.788721 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f059a408-6e5e-4308-a4a0-905711da3160 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:46:50.871358 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f059a408-6e5e-4308-a4a0-905711da3160 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:46:50.872921 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f059a408-6e5e-4308-a4a0-905711da3160 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:46:50.884399 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1376/2791] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:46:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:46:54.678965 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b0b9a3-4465-4fb6-ab2d-026668a3049f service nova] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:46:54.693196 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b0b9a3-4465-4fb6-ab2d-026668a3049f service nova] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:46:54.699121 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef85ec-1e6a-4d10-adc0-e150b4590eb9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef1d49b36cdc4665b82487be97e9dcd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:46:54.700034 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef85ec-1e6a-4d10-adc0-e150b4590eb9 service placement] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:46:54.700464 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef85ec-1e6a-4d10-adc0-e150b4590eb9 service placement] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:46:54.700864 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef85ec-1e6a-4d10-adc0-e150b4590eb9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:46:54.704010 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-69ef85ec-1e6a-4d10-adc0-e150b4590eb9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:46:54.717426 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ef85ec-1e6a-4d10-adc0-e150b4590eb9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:46:54.719123 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ef85ec-1e6a-4d10-adc0-e150b4590eb9 service placement] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:46:54.732440 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1416/2792] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:46:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:46:54.795080 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f059a408-6e5e-4308-a4a0-905711da3160 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:46:54.796721 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f059a408-6e5e-4308-a4a0-905711da3160 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:46:54.810462 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f059a408-6e5e-4308-a4a0-905711da3160 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:46:54.823818 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee201c3-ec1a-4ead-bb4f-d89b594b67a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:46:54.823818 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee201c3-ec1a-4ead-bb4f-d89b594b67a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:46:54.823818 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee201c3-ec1a-4ead-bb4f-d89b594b67a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:46:54.823818 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee201c3-ec1a-4ead-bb4f-d89b594b67a6 admin admin] PATH_INFO: `/v3/users/93785710c10749f9aa3d29b43965909c` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:46:54.825244 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee201c3-ec1a-4ead-bb4f-d89b594b67a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93785710c10749f9aa3d29b43965909c)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:46:54.827356 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee201c3-ec1a-4ead-bb4f-d89b594b67a6 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:46:54.886830 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-4ee201c3-ec1a-4ead-bb4f-d89b594b67a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93785710c10749f9aa3d29b43965909c'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:46:54.897450 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-4ee201c3-ec1a-4ead-bb4f-d89b594b67a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93785710c10749f9aa3d29b43965909c'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:46:54.898869 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-4ee201c3-ec1a-4ead-bb4f-d89b594b67a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93785710c10749f9aa3d29b43965909c'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:46:54.900135 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-4ee201c3-ec1a-4ead-bb4f-d89b594b67a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93785710c10749f9aa3d29b43965909c'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:46:54.917899 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-4ee201c3-ec1a-4ead-bb4f-d89b594b67a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93785710c10749f9aa3d29b43965909c'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:46:54.935167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1377/2793] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:46:54 2022] DELETE /identity/v3/users/93785710c10749f9aa3d29b43965909c => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:46:55.188566 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef85ec-1e6a-4d10-adc0-e150b4590eb9 service placement] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:46:55.204197 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef85ec-1e6a-4d10-adc0-e150b4590eb9 service placement] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:46:55.284155 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214b010-b567-4549-ae07-aa8f07141019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:46:55.285757 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214b010-b567-4549-ae07-aa8f07141019 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:46:55.286261 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214b010-b567-4549-ae07-aa8f07141019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:46:55.286758 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214b010-b567-4549-ae07-aa8f07141019 admin admin] PATH_INFO: `/v3/projects/21a4c114c12d40abb896e2637ee6ab83` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:46:55.289138 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3214b010-b567-4549-ae07-aa8f07141019 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21a4c114c12d40abb896e2637ee6ab83)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:46:55.292832 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3214b010-b567-4549-ae07-aa8f07141019 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:46:55.374781 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3214b010-b567-4549-ae07-aa8f07141019 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21a4c114c12d40abb896e2637ee6ab83', 'request_id': 'req-3214b010-b567-4549-ae07-aa8f07141019'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:46:55.382122 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3214b010-b567-4549-ae07-aa8f07141019 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21a4c114c12d40abb896e2637ee6ab83', 'request_id': 'req-3214b010-b567-4549-ae07-aa8f07141019'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:46:55.395984 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3214b010-b567-4549-ae07-aa8f07141019 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21a4c114c12d40abb896e2637ee6ab83', 'request_id': 'req-3214b010-b567-4549-ae07-aa8f07141019'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:46:55.411744 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3214b010-b567-4549-ae07-aa8f07141019 admin admin] The token cache is being invalidate because project 21a4c114c12d40abb896e2637ee6ab83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:46:55.412106 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3214b010-b567-4549-ae07-aa8f07141019 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:46:55.412798 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3214b010-b567-4549-ae07-aa8f07141019 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:46:55.414550 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1417/2794] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:46:55 2022] DELETE /identity/v3/projects/21a4c114c12d40abb896e2637ee6ab83 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:46:55.419942 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4642b4-3fc1-466f-8de1-2bd89c8e5c0d None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:46:55.420237 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4642b4-3fc1-466f-8de1-2bd89c8e5c0d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:46:55.420521 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4642b4-3fc1-466f-8de1-2bd89c8e5c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:46:55.442748 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-ef4642b4-3fc1-466f-8de1-2bd89c8e5c0d None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:46:55.444224 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ef4642b4-3fc1-466f-8de1-2bd89c8e5c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:46:55.533349 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1378/2795] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:46:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:46:55.543113 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86f0d3-25ef-4db8-a498-b04f8ef995ae None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:46:55.543553 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86f0d3-25ef-4db8-a498-b04f8ef995ae None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:46:55.544032 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86f0d3-25ef-4db8-a498-b04f8ef995ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:46:55.567091 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-cb86f0d3-25ef-4db8-a498-b04f8ef995ae None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:46:55.568453 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-cb86f0d3-25ef-4db8-a498-b04f8ef995ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:46:55.577623 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1418/2796] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:46:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:46:55.583000 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4642b4-3fc1-466f-8de1-2bd89c8e5c0d None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:46:55.596432 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4642b4-3fc1-466f-8de1-2bd89c8e5c0d None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:46:55.602932 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461b326-445b-41a9-924e-55ba1b8e81cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:46:55.603641 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461b326-445b-41a9-924e-55ba1b8e81cf admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:46:55.603925 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461b326-445b-41a9-924e-55ba1b8e81cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:46:55.604204 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461b326-445b-41a9-924e-55ba1b8e81cf admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:46:55.604952 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e461b326-445b-41a9-924e-55ba1b8e81cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:46:55.605262 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e461b326-445b-41a9-924e-55ba1b8e81cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:46:55.606948 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e461b326-445b-41a9-924e-55ba1b8e81cf admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:46:55.608927 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:46:55.624365 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1379/2797] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:46:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:46:55.628310 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb86f0d3-25ef-4db8-a498-b04f8ef995ae None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:46:55.639211 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb86f0d3-25ef-4db8-a498-b04f8ef995ae None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:46:55.644767 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f31bd5-e589-4527-90b0-de11134d9ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:46:55.645605 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f31bd5-e589-4527-90b0-de11134d9ca8 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:46:55.645983 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f31bd5-e589-4527-90b0-de11134d9ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:46:55.646345 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f31bd5-e589-4527-90b0-de11134d9ca8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:46:55.647102 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f31bd5-e589-4527-90b0-de11134d9ca8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:46:55.649192 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f31bd5-e589-4527-90b0-de11134d9ca8 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:46:55.694864 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1419/2798] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:46:55 2022] POST /identity/v3/projects => generated 373 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:46:55.701046 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461b326-445b-41a9-924e-55ba1b8e81cf admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:46:55.716756 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461b326-445b-41a9-924e-55ba1b8e81cf admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:46:55.791924 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d5232-9ed3-4415-b8d4-3583c3a96388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:46:55.792799 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d5232-9ed3-4415-b8d4-3583c3a96388 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:46:55.793199 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d5232-9ed3-4415-b8d4-3583c3a96388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:46:55.793525 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d5232-9ed3-4415-b8d4-3583c3a96388 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:46:55.794379 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21d5232-9ed3-4415-b8d4-3583c3a96388 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:46:55.796610 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21d5232-9ed3-4415-b8d4-3583c3a96388 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:46:55.821706 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1380/2799] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:46:55 2022] POST /identity/v3/users => generated 287 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:46:55.827447 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f31bd5-e589-4527-90b0-de11134d9ca8 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:46:55.840062 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f31bd5-e589-4527-90b0-de11134d9ca8 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:46:55.845420 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fee4094-5e40-4bc2-8b4d-3465cf1a7fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:46:55.846267 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fee4094-5e40-4bc2-8b4d-3465cf1a7fc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:46:55.846552 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fee4094-5e40-4bc2-8b4d-3465cf1a7fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:46:55.846830 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fee4094-5e40-4bc2-8b4d-3465cf1a7fc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:46:55.847589 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fee4094-5e40-4bc2-8b4d-3465cf1a7fc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:46:55.849433 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fee4094-5e40-4bc2-8b4d-3465cf1a7fc3 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:46:55.862652 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1420/2800] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:46:55.868831 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d5232-9ed3-4415-b8d4-3583c3a96388 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:46:55.880198 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d5232-9ed3-4415-b8d4-3583c3a96388 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:46:55.885796 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84514b7-fa34-44bd-b86e-750ac34c5f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:46:55.886498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84514b7-fa34-44bd-b86e-750ac34c5f2f admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:46:55.886787 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84514b7-fa34-44bd-b86e-750ac34c5f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:46:55.887068 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84514b7-fa34-44bd-b86e-750ac34c5f2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:46:55.887811 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84514b7-fa34-44bd-b86e-750ac34c5f2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:46:55.889464 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84514b7-fa34-44bd-b86e-750ac34c5f2f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:46:55.901658 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1381/2801] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:46:55.906722 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fee4094-5e40-4bc2-8b4d-3465cf1a7fc3 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:46:55.918008 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fee4094-5e40-4bc2-8b4d-3465cf1a7fc3 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:46:55.923032 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f70d350-7fe3-4451-b317-bce2902cb406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:46:55.924036 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f70d350-7fe3-4451-b317-bce2902cb406 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:46:55.924483 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f70d350-7fe3-4451-b317-bce2902cb406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:46:55.924780 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f70d350-7fe3-4451-b317-bce2902cb406 admin admin] PATH_INFO: `/v3/projects/7f848061159947759a8e8058ad9e69b5/users/564af2d5973546649c565ce39674bbd4/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:46:55.939850 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:46:55.945093 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f70d350-7fe3-4451-b317-bce2902cb406 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f848061159947759a8e8058ad9e69b5, user_id=564af2d5973546649c565ce39674bbd4, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:46:55.947954 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f70d350-7fe3-4451-b317-bce2902cb406 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:46:55.968162 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1421/2802] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:46:55 2022] PUT /identity/v3/projects/7f848061159947759a8e8058ad9e69b5/users/564af2d5973546649c565ce39674bbd4/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:46:55.982005 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da092b34-5dc5-4e52-9a52-8108ca0a217c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:46:55.982522 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da092b34-5dc5-4e52-9a52-8108ca0a217c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:46:55.983065 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da092b34-5dc5-4e52-9a52-8108ca0a217c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:46:56.030564 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-da092b34-5dc5-4e52-9a52-8108ca0a217c None None] MFA Rules not processed for user `564af2d5973546649c565ce39674bbd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:46:56.032470 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-da092b34-5dc5-4e52-9a52-8108ca0a217c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:46:56.151388 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1382/2803] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:46:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:46:56.156413 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f70d350-7fe3-4451-b317-bce2902cb406 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:46:56.246493 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f70d350-7fe3-4451-b317-bce2902cb406 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:46:56.251449 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98cd73-be56-4cca-8fee-6c4aaa0eadb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:46:56.252127 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98cd73-be56-4cca-8fee-6c4aaa0eadb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:46:56.252405 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98cd73-be56-4cca-8fee-6c4aaa0eadb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:46:56.252688 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98cd73-be56-4cca-8fee-6c4aaa0eadb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:46:56.253387 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac98cd73-be56-4cca-8fee-6c4aaa0eadb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:46:56.255797 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac98cd73-be56-4cca-8fee-6c4aaa0eadb9 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:46:56.288132 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1422/2804] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:46:56 2022] POST /identity/v3/projects => generated 373 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:46:56.292593 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da092b34-5dc5-4e52-9a52-8108ca0a217c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:46:56.304601 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da092b34-5dc5-4e52-9a52-8108ca0a217c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:46:56.374811 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b154edc-52e6-49a6-8cc9-9ae5636492be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:46:56.375506 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b154edc-52e6-49a6-8cc9-9ae5636492be admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:46:56.375801 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b154edc-52e6-49a6-8cc9-9ae5636492be admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:46:56.376086 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b154edc-52e6-49a6-8cc9-9ae5636492be admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:46:56.376806 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b154edc-52e6-49a6-8cc9-9ae5636492be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:46:56.378804 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b154edc-52e6-49a6-8cc9-9ae5636492be admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:46:56.388137 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:46:56.404023 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1383/2805] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:46:56 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:46:56.409001 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98cd73-be56-4cca-8fee-6c4aaa0eadb9 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:46:56.421338 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98cd73-be56-4cca-8fee-6c4aaa0eadb9 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:46:56.427440 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bb163f-2df9-4104-a8af-cae6526888a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:46:56.428194 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb163f-2df9-4104-a8af-cae6526888a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:46:56.428490 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb163f-2df9-4104-a8af-cae6526888a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:46:56.428776 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb163f-2df9-4104-a8af-cae6526888a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:46:56.429568 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bb163f-2df9-4104-a8af-cae6526888a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:46:56.431755 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bb163f-2df9-4104-a8af-cae6526888a7 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:46:56.444840 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1423/2806] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:46:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:46:56.449485 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b154edc-52e6-49a6-8cc9-9ae5636492be admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:46:56.461542 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b154edc-52e6-49a6-8cc9-9ae5636492be admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:46:56.466944 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994d6105-656b-4ee2-9153-153a2dfdb509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:46:56.467868 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d6105-656b-4ee2-9153-153a2dfdb509 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:46:56.468145 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d6105-656b-4ee2-9153-153a2dfdb509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:46:56.468434 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d6105-656b-4ee2-9153-153a2dfdb509 admin admin] PATH_INFO: `/v3/projects/31c65e720e7645f9bac9a26803620585/users/a8e778e8724e4a2093fcaca5649d8164/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:46:56.486401 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994d6105-656b-4ee2-9153-153a2dfdb509 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31c65e720e7645f9bac9a26803620585, user_id=a8e778e8724e4a2093fcaca5649d8164, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:46:56.490383 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994d6105-656b-4ee2-9153-153a2dfdb509 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:46:56.508330 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1384/2807] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:46:56 2022] PUT /identity/v3/projects/31c65e720e7645f9bac9a26803620585/users/a8e778e8724e4a2093fcaca5649d8164/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:46:56.512620 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bb163f-2df9-4104-a8af-cae6526888a7 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:46:56.523585 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bb163f-2df9-4104-a8af-cae6526888a7 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:46:56.592218 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd24eac-d02c-42eb-b969-f99074d9d0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:46:56.592907 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd24eac-d02c-42eb-b969-f99074d9d0cb admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:46:56.593184 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd24eac-d02c-42eb-b969-f99074d9d0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:46:56.593438 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd24eac-d02c-42eb-b969-f99074d9d0cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:46:56.594215 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd24eac-d02c-42eb-b969-f99074d9d0cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:46:56.595859 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd24eac-d02c-42eb-b969-f99074d9d0cb admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:46:56.608030 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1424/2808] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:46:56 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:46:56.612691 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994d6105-656b-4ee2-9153-153a2dfdb509 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:46:56.624113 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994d6105-656b-4ee2-9153-153a2dfdb509 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:46:56.629041 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c5d0f9-06bc-4f4d-8875-277c523fb51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:46:56.630061 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c5d0f9-06bc-4f4d-8875-277c523fb51f admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:46:56.630346 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c5d0f9-06bc-4f4d-8875-277c523fb51f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:46:56.630820 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c5d0f9-06bc-4f4d-8875-277c523fb51f admin admin] PATH_INFO: `/v3/projects/31c65e720e7645f9bac9a26803620585/users/a8e778e8724e4a2093fcaca5649d8164/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:46:56.632944 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c5d0f9-06bc-4f4d-8875-277c523fb51f admin admin] RBAC: Authorizing `identity:create_grant(project_id=31c65e720e7645f9bac9a26803620585, user_id=a8e778e8724e4a2093fcaca5649d8164, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:46:56.635756 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c5d0f9-06bc-4f4d-8875-277c523fb51f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:46:56.648198 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1385/2809] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:46:56 2022] PUT /identity/v3/projects/31c65e720e7645f9bac9a26803620585/users/a8e778e8724e4a2093fcaca5649d8164/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:46:56.659022 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a45b8-e114-4f66-a4e5-d0fa0fcec8cf None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:46:56.659364 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a45b8-e114-4f66-a4e5-d0fa0fcec8cf None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:46:56.659557 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a45b8-e114-4f66-a4e5-d0fa0fcec8cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:46:56.697515 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-b78a45b8-e114-4f66-a4e5-d0fa0fcec8cf None None] MFA Rules not processed for user `a8e778e8724e4a2093fcaca5649d8164`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:46:56.698805 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b78a45b8-e114-4f66-a4e5-d0fa0fcec8cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:46:56.766710 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:46:56.792069 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1425/2810] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:46:56 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:46:56.808791 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c5d0f9-06bc-4f4d-8875-277c523fb51f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:46:56.810888 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-83c5d0f9-06bc-4f4d-8875-277c523fb51f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:46:56.880078 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c5d0f9-06bc-4f4d-8875-277c523fb51f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:46:56.884507 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ebaa54-d829-4ab3-adcb-fb9732afbf52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:46:56.885197 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ebaa54-d829-4ab3-adcb-fb9732afbf52 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:46:56.885491 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ebaa54-d829-4ab3-adcb-fb9732afbf52 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:46:56.885885 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ebaa54-d829-4ab3-adcb-fb9732afbf52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:46:56.896553 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ebaa54-d829-4ab3-adcb-fb9732afbf52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:46:56.898596 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ebaa54-d829-4ab3-adcb-fb9732afbf52 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:46:56.909323 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1386/2811] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:46:56 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:46:57.367314 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a45b8-e114-4f66-a4e5-d0fa0fcec8cf None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:46:57.442017 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a45b8-e114-4f66-a4e5-d0fa0fcec8cf None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:46:57.447063 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d69d4e-18c9-42aa-9088-b880be9f2c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:46:57.448162 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d69d4e-18c9-42aa-9088-b880be9f2c66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:46:57.448451 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d69d4e-18c9-42aa-9088-b880be9f2c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:46:57.448735 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d69d4e-18c9-42aa-9088-b880be9f2c66 admin admin] PATH_INFO: `/v3/users/564af2d5973546649c565ce39674bbd4` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:46:57.450183 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d69d4e-18c9-42aa-9088-b880be9f2c66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=564af2d5973546649c565ce39674bbd4)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:46:57.452173 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d69d4e-18c9-42aa-9088-b880be9f2c66 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:46:57.509887 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-19d69d4e-18c9-42aa-9088-b880be9f2c66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '564af2d5973546649c565ce39674bbd4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:46:57.524626 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-19d69d4e-18c9-42aa-9088-b880be9f2c66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '564af2d5973546649c565ce39674bbd4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:46:57.525661 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-19d69d4e-18c9-42aa-9088-b880be9f2c66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '564af2d5973546649c565ce39674bbd4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:46:57.526444 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-19d69d4e-18c9-42aa-9088-b880be9f2c66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '564af2d5973546649c565ce39674bbd4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:46:57.537549 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-19d69d4e-18c9-42aa-9088-b880be9f2c66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '564af2d5973546649c565ce39674bbd4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:46:57.555236 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1426/2812] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:46:57 2022] DELETE /identity/v3/users/564af2d5973546649c565ce39674bbd4 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:46:57.559817 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeecccc-a67c-44e8-b6c9-7d37cbde8c0b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:46:57.560113 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeecccc-a67c-44e8-b6c9-7d37cbde8c0b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:46:57.560399 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeecccc-a67c-44e8-b6c9-7d37cbde8c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:46:57.580162 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-dbeecccc-a67c-44e8-b6c9-7d37cbde8c0b None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:46:57.581510 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-dbeecccc-a67c-44e8-b6c9-7d37cbde8c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:46:57.662802 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1387/2813] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:46:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:46:57.676945 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d69d4e-18c9-42aa-9088-b880be9f2c66 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:46:57.678622 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-19d69d4e-18c9-42aa-9088-b880be9f2c66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:46:57.753110 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d69d4e-18c9-42aa-9088-b880be9f2c66 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:46:57.759660 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef2395a-f456-403f-9132-58437fd36662 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:46:57.760537 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2395a-f456-403f-9132-58437fd36662 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:46:57.760843 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2395a-f456-403f-9132-58437fd36662 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:46:57.761121 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2395a-f456-403f-9132-58437fd36662 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:46:57.773397 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef2395a-f456-403f-9132-58437fd36662 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:46:57.775256 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef2395a-f456-403f-9132-58437fd36662 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:46:57.788622 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1427/2814] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:46:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:46:58.127686 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeecccc-a67c-44e8-b6c9-7d37cbde8c0b None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:46:58.130506 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-dbeecccc-a67c-44e8-b6c9-7d37cbde8c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:46:58.150000 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeecccc-a67c-44e8-b6c9-7d37cbde8c0b None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:46:58.155810 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc6615e-65c8-4baa-ab5e-6706016eddac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:46:58.156723 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6615e-65c8-4baa-ab5e-6706016eddac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:46:58.157006 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6615e-65c8-4baa-ab5e-6706016eddac admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:46:58.157285 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6615e-65c8-4baa-ab5e-6706016eddac admin admin] PATH_INFO: `/v3/projects/7f848061159947759a8e8058ad9e69b5` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:46:58.158946 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc6615e-65c8-4baa-ab5e-6706016eddac admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f848061159947759a8e8058ad9e69b5)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:46:58.161801 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc6615e-65c8-4baa-ab5e-6706016eddac admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:46:58.227131 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-bcc6615e-65c8-4baa-ab5e-6706016eddac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f848061159947759a8e8058ad9e69b5', 'request_id': 'req-bcc6615e-65c8-4baa-ab5e-6706016eddac'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:46:58.230155 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:INSERT=5,keystone:SELECT=91,keystone:DELETE=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:46:58.238016 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-bcc6615e-65c8-4baa-ab5e-6706016eddac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f848061159947759a8e8058ad9e69b5', 'request_id': 'req-bcc6615e-65c8-4baa-ab5e-6706016eddac'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:46:58.247105 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-bcc6615e-65c8-4baa-ab5e-6706016eddac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f848061159947759a8e8058ad9e69b5', 'request_id': 'req-bcc6615e-65c8-4baa-ab5e-6706016eddac'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:46:58.260199 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-bcc6615e-65c8-4baa-ab5e-6706016eddac admin admin] The token cache is being invalidate because project 7f848061159947759a8e8058ad9e69b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:46:58.260644 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-bcc6615e-65c8-4baa-ab5e-6706016eddac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:46:58.261507 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-bcc6615e-65c8-4baa-ab5e-6706016eddac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:46:58.263436 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1388/2815] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:46:58 2022] DELETE /identity/v3/projects/7f848061159947759a8e8058ad9e69b5 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:46:58.267952 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef2395a-f456-403f-9132-58437fd36662 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:46:58.282044 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef2395a-f456-403f-9132-58437fd36662 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:46:58.289115 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:46:58.361279 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145ca363-250a-4978-a213-7451898324d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:46:58.362949 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ca363-250a-4978-a213-7451898324d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:46:58.363401 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ca363-250a-4978-a213-7451898324d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:46:58.364261 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ca363-250a-4978-a213-7451898324d9 admin admin] PATH_INFO: `/v3/users/a8e778e8724e4a2093fcaca5649d8164` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:46:58.366054 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145ca363-250a-4978-a213-7451898324d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8e778e8724e4a2093fcaca5649d8164)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:46:58.368148 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145ca363-250a-4978-a213-7451898324d9 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:46:58.432231 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-145ca363-250a-4978-a213-7451898324d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8e778e8724e4a2093fcaca5649d8164'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:46:58.446845 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-145ca363-250a-4978-a213-7451898324d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8e778e8724e4a2093fcaca5649d8164'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:46:58.447604 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-145ca363-250a-4978-a213-7451898324d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8e778e8724e4a2093fcaca5649d8164'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:46:58.448267 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-145ca363-250a-4978-a213-7451898324d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8e778e8724e4a2093fcaca5649d8164'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:46:58.459220 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-145ca363-250a-4978-a213-7451898324d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8e778e8724e4a2093fcaca5649d8164'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:46:58.475538 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1428/2816] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:46:58 2022] DELETE /identity/v3/users/a8e778e8724e4a2093fcaca5649d8164 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:46:58.841480 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc6615e-65c8-4baa-ab5e-6706016eddac admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:46:58.843176 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-bcc6615e-65c8-4baa-ab5e-6706016eddac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:46:58.858891 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc6615e-65c8-4baa-ab5e-6706016eddac admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:46:58.934883 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d923ab79-de11-450a-9019-844c9585db66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:46:58.935868 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923ab79-de11-450a-9019-844c9585db66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:46:58.936277 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923ab79-de11-450a-9019-844c9585db66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:46:58.936674 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923ab79-de11-450a-9019-844c9585db66 admin admin] PATH_INFO: `/v3/projects/31c65e720e7645f9bac9a26803620585` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:46:58.938130 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d923ab79-de11-450a-9019-844c9585db66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31c65e720e7645f9bac9a26803620585)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:46:58.940340 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d923ab79-de11-450a-9019-844c9585db66 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:46:58.999026 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d923ab79-de11-450a-9019-844c9585db66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31c65e720e7645f9bac9a26803620585', 'request_id': 'req-d923ab79-de11-450a-9019-844c9585db66'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:46:59.009835 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d923ab79-de11-450a-9019-844c9585db66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31c65e720e7645f9bac9a26803620585', 'request_id': 'req-d923ab79-de11-450a-9019-844c9585db66'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:46:59.016694 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d923ab79-de11-450a-9019-844c9585db66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31c65e720e7645f9bac9a26803620585', 'request_id': 'req-d923ab79-de11-450a-9019-844c9585db66'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:46:59.029774 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d923ab79-de11-450a-9019-844c9585db66 admin admin] The token cache is being invalidate because project 31c65e720e7645f9bac9a26803620585 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:46:59.030116 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d923ab79-de11-450a-9019-844c9585db66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:46:59.031009 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d923ab79-de11-450a-9019-844c9585db66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:46:59.032677 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1389/2817] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:46:58 2022] DELETE /identity/v3/projects/31c65e720e7645f9bac9a26803620585 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:46:59.039641 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f260c5b-c0b7-4985-a9c1-7aa4cd898d8f None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:46:59.040416 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f260c5b-c0b7-4985-a9c1-7aa4cd898d8f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:46:59.040416 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f260c5b-c0b7-4985-a9c1-7aa4cd898d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:46:59.060423 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-5f260c5b-c0b7-4985-a9c1-7aa4cd898d8f None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:46:59.061711 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5f260c5b-c0b7-4985-a9c1-7aa4cd898d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:46:59.140070 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1429/2818] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:46:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:46:59.149029 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f6707-acd8-4e7d-9ab8-71f2c90feb14 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:46:59.149959 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f6707-acd8-4e7d-9ab8-71f2c90feb14 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:46:59.150148 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f6707-acd8-4e7d-9ab8-71f2c90feb14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:46:59.171542 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-eb8f6707-acd8-4e7d-9ab8-71f2c90feb14 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:46:59.172802 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-eb8f6707-acd8-4e7d-9ab8-71f2c90feb14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:46:59.182524 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1390/2819] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:46:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:46:59.187840 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f260c5b-c0b7-4985-a9c1-7aa4cd898d8f None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:46:59.200031 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f260c5b-c0b7-4985-a9c1-7aa4cd898d8f None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:46:59.205828 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c008cf-1c61-4011-86c2-64b6c43d05af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:46:59.206550 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c008cf-1c61-4011-86c2-64b6c43d05af admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:46:59.206899 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c008cf-1c61-4011-86c2-64b6c43d05af admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:46:59.207567 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c008cf-1c61-4011-86c2-64b6c43d05af admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:46:59.208546 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c008cf-1c61-4011-86c2-64b6c43d05af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:46:59.208849 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c008cf-1c61-4011-86c2-64b6c43d05af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:46:59.210735 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c008cf-1c61-4011-86c2-64b6c43d05af admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:46:59.232829 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1430/2820] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:46:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:46:59.237675 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f6707-acd8-4e7d-9ab8-71f2c90feb14 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:46:59.249590 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f6707-acd8-4e7d-9ab8-71f2c90feb14 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:46:59.255793 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b93138f-441f-4b64-bcb7-d9235538d33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:46:59.256521 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b93138f-441f-4b64-bcb7-d9235538d33e admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:46:59.256808 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b93138f-441f-4b64-bcb7-d9235538d33e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:46:59.257167 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b93138f-441f-4b64-bcb7-d9235538d33e admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:46:59.258249 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b93138f-441f-4b64-bcb7-d9235538d33e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:46:59.260252 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b93138f-441f-4b64-bcb7-d9235538d33e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:46:59.305036 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1391/2821] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:46:59 2022] POST /identity/v3/projects => generated 373 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:46:59.309171 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c008cf-1c61-4011-86c2-64b6c43d05af admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:46:59.320518 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c008cf-1c61-4011-86c2-64b6c43d05af admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:46:59.339060 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:46:59.397541 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf8ac8e-ceb7-472a-82c3-25c4d65a62f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:46:59.398297 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8ac8e-ceb7-472a-82c3-25c4d65a62f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:46:59.398582 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8ac8e-ceb7-472a-82c3-25c4d65a62f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:46:59.398914 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8ac8e-ceb7-472a-82c3-25c4d65a62f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:46:59.400007 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf8ac8e-ceb7-472a-82c3-25c4d65a62f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:46:59.402002 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf8ac8e-ceb7-472a-82c3-25c4d65a62f7 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:46:59.426937 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1431/2822] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:46:59 2022] POST /identity/v3/users => generated 287 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:46:59.431457 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b93138f-441f-4b64-bcb7-d9235538d33e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:46:59.443981 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b93138f-441f-4b64-bcb7-d9235538d33e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:46:59.449065 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cef8098-162f-448e-9a24-dcefa48a5237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:46:59.449842 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef8098-162f-448e-9a24-dcefa48a5237 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:46:59.450132 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef8098-162f-448e-9a24-dcefa48a5237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:46:59.450412 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef8098-162f-448e-9a24-dcefa48a5237 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:46:59.451185 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cef8098-162f-448e-9a24-dcefa48a5237 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:46:59.453034 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cef8098-162f-448e-9a24-dcefa48a5237 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:46:59.466236 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1392/2823] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:46:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:46:59.470643 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf8ac8e-ceb7-472a-82c3-25c4d65a62f7 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:46:59.483317 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf8ac8e-ceb7-472a-82c3-25c4d65a62f7 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:46:59.488329 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eae169-9e33-47e8-8a11-a8c72e152625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:46:59.489875 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eae169-9e33-47e8-8a11-a8c72e152625 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:46:59.490170 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eae169-9e33-47e8-8a11-a8c72e152625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:46:59.490593 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eae169-9e33-47e8-8a11-a8c72e152625 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:46:59.491354 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eae169-9e33-47e8-8a11-a8c72e152625 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:46:59.493162 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eae169-9e33-47e8-8a11-a8c72e152625 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:46:59.505518 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1432/2824] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:46:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:46:59.510863 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cef8098-162f-448e-9a24-dcefa48a5237 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:46:59.522259 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cef8098-162f-448e-9a24-dcefa48a5237 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:46:59.527761 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633bc471-5f1d-4ea7-bfe2-65eb2e54bdf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:46:59.528679 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633bc471-5f1d-4ea7-bfe2-65eb2e54bdf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:46:59.528964 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633bc471-5f1d-4ea7-bfe2-65eb2e54bdf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:46:59.529236 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633bc471-5f1d-4ea7-bfe2-65eb2e54bdf5 admin admin] PATH_INFO: `/v3/projects/673fc97b64454b2cb8d36e4e6a473a82/users/ab2481aaa3ac451fa31bc2e7c54f6b56/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:46:59.547522 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633bc471-5f1d-4ea7-bfe2-65eb2e54bdf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=673fc97b64454b2cb8d36e4e6a473a82, user_id=ab2481aaa3ac451fa31bc2e7c54f6b56, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:46:59.550889 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633bc471-5f1d-4ea7-bfe2-65eb2e54bdf5 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:46:59.568710 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1393/2825] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:46:59 2022] PUT /identity/v3/projects/673fc97b64454b2cb8d36e4e6a473a82/users/ab2481aaa3ac451fa31bc2e7c54f6b56/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:46:59.581894 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d969d7b-a98d-40ae-a357-651260ffd1f9 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:46:59.582710 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d969d7b-a98d-40ae-a357-651260ffd1f9 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:46:59.582944 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d969d7b-a98d-40ae-a357-651260ffd1f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:46:59.617804 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-8d969d7b-a98d-40ae-a357-651260ffd1f9 None None] MFA Rules not processed for user `ab2481aaa3ac451fa31bc2e7c54f6b56`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:46:59.619445 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8d969d7b-a98d-40ae-a357-651260ffd1f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:46:59.731807 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1433/2826] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:46:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:46:59.746952 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633bc471-5f1d-4ea7-bfe2-65eb2e54bdf5 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:46:59.749082 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-633bc471-5f1d-4ea7-bfe2-65eb2e54bdf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:46:59.762772 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:46:59.848701 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633bc471-5f1d-4ea7-bfe2-65eb2e54bdf5 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:46:59.853356 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e501e-b330-4a64-ad98-6cd893f063c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:46:59.854519 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e501e-b330-4a64-ad98-6cd893f063c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:46:59.855013 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e501e-b330-4a64-ad98-6cd893f063c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:46:59.855495 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e501e-b330-4a64-ad98-6cd893f063c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:46:59.869596 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5e501e-b330-4a64-ad98-6cd893f063c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:46:59.870406 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5e501e-b330-4a64-ad98-6cd893f063c7 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:46:59.882180 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1394/2827] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:46:59 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:46:59.904113 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d969d7b-a98d-40ae-a357-651260ffd1f9 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:00.019396 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d969d7b-a98d-40ae-a357-651260ffd1f9 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:00.027949 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1d899b-53ca-4f56-8c09-a0d5f6468d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:00.029488 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d899b-53ca-4f56-8c09-a0d5f6468d6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:00.030000 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d899b-53ca-4f56-8c09-a0d5f6468d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:00.030482 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d899b-53ca-4f56-8c09-a0d5f6468d6c admin admin] PATH_INFO: `/v3/users/ab2481aaa3ac451fa31bc2e7c54f6b56` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:00.033005 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1d899b-53ca-4f56-8c09-a0d5f6468d6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab2481aaa3ac451fa31bc2e7c54f6b56)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:00.037076 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1d899b-53ca-4f56-8c09-a0d5f6468d6c admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:00.090181 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:47:00.118412 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-af1d899b-53ca-4f56-8c09-a0d5f6468d6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab2481aaa3ac451fa31bc2e7c54f6b56'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:00.141029 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-af1d899b-53ca-4f56-8c09-a0d5f6468d6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab2481aaa3ac451fa31bc2e7c54f6b56'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:00.142262 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-af1d899b-53ca-4f56-8c09-a0d5f6468d6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab2481aaa3ac451fa31bc2e7c54f6b56'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:00.143412 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-af1d899b-53ca-4f56-8c09-a0d5f6468d6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab2481aaa3ac451fa31bc2e7c54f6b56'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:00.156878 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-af1d899b-53ca-4f56-8c09-a0d5f6468d6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab2481aaa3ac451fa31bc2e7c54f6b56'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:00.174006 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1434/2828] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:46:59 2022] DELETE /identity/v3/users/ab2481aaa3ac451fa31bc2e7c54f6b56 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:47:00.180008 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6156722-4cf8-45ac-96f1-904926670af8 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:00.180759 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6156722-4cf8-45ac-96f1-904926670af8 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:00.180759 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6156722-4cf8-45ac-96f1-904926670af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:00.200876 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-e6156722-4cf8-45ac-96f1-904926670af8 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:47:00.202169 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e6156722-4cf8-45ac-96f1-904926670af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:47:00.275201 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1395/2829] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:47:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:47:00.285669 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1d899b-53ca-4f56-8c09-a0d5f6468d6c admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:00.287885 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-af1d899b-53ca-4f56-8c09-a0d5f6468d6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:47:00.374651 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1d899b-53ca-4f56-8c09-a0d5f6468d6c admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:00.381001 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b0fbe8-baae-48bc-ac3b-27ddb9f961f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:00.382202 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b0fbe8-baae-48bc-ac3b-27ddb9f961f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:00.382664 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b0fbe8-baae-48bc-ac3b-27ddb9f961f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:00.383159 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b0fbe8-baae-48bc-ac3b-27ddb9f961f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:00.399281 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b0fbe8-baae-48bc-ac3b-27ddb9f961f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:00.400921 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b0fbe8-baae-48bc-ac3b-27ddb9f961f2 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:00.412702 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1435/2830] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:47:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:47:00.795725 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6156722-4cf8-45ac-96f1-904926670af8 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:00.797549 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e6156722-4cf8-45ac-96f1-904926670af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:47:00.821337 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6156722-4cf8-45ac-96f1-904926670af8 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:00.828390 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6c51db-bb14-41bb-98bf-bac8613f9176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:00.830589 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c51db-bb14-41bb-98bf-bac8613f9176 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:00.831312 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c51db-bb14-41bb-98bf-bac8613f9176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:00.831958 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c51db-bb14-41bb-98bf-bac8613f9176 admin admin] PATH_INFO: `/v3/projects/673fc97b64454b2cb8d36e4e6a473a82` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:00.835178 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6c51db-bb14-41bb-98bf-bac8613f9176 admin admin] RBAC: Authorizing `identity:delete_project(project_id=673fc97b64454b2cb8d36e4e6a473a82)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:00.839537 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6c51db-bb14-41bb-98bf-bac8613f9176 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:00.902170 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-cd6c51db-bb14-41bb-98bf-bac8613f9176 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '673fc97b64454b2cb8d36e4e6a473a82', 'request_id': 'req-cd6c51db-bb14-41bb-98bf-bac8613f9176'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:00.912573 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-cd6c51db-bb14-41bb-98bf-bac8613f9176 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '673fc97b64454b2cb8d36e4e6a473a82', 'request_id': 'req-cd6c51db-bb14-41bb-98bf-bac8613f9176'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:00.922498 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-cd6c51db-bb14-41bb-98bf-bac8613f9176 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '673fc97b64454b2cb8d36e4e6a473a82', 'request_id': 'req-cd6c51db-bb14-41bb-98bf-bac8613f9176'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:00.944013 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-cd6c51db-bb14-41bb-98bf-bac8613f9176 admin admin] The token cache is being invalidate because project 673fc97b64454b2cb8d36e4e6a473a82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:47:00.944520 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-cd6c51db-bb14-41bb-98bf-bac8613f9176 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:00.945598 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-cd6c51db-bb14-41bb-98bf-bac8613f9176 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:00.948442 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1396/2831] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:47:00 2022] DELETE /identity/v3/projects/673fc97b64454b2cb8d36e4e6a473a82 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:47:00.955423 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f031c-ea6e-4ba5-acf0-4239aec34986 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:00.955723 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f031c-ea6e-4ba5-acf0-4239aec34986 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:00.955977 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f031c-ea6e-4ba5-acf0-4239aec34986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:00.978799 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-865f031c-ea6e-4ba5-acf0-4239aec34986 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:47:00.980093 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-865f031c-ea6e-4ba5-acf0-4239aec34986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:47:01.069131 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1436/2832] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:47:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:47:01.077441 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87261841-e9a1-4533-977e-d24e5c87e2f3 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:01.077767 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87261841-e9a1-4533-977e-d24e5c87e2f3 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:01.078123 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87261841-e9a1-4533-977e-d24e5c87e2f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:01.101419 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-87261841-e9a1-4533-977e-d24e5c87e2f3 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:47:01.103418 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-87261841-e9a1-4533-977e-d24e5c87e2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:47:01.113561 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1397/2833] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:47:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:47:01.120288 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f031c-ea6e-4ba5-acf0-4239aec34986 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:01.131713 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f031c-ea6e-4ba5-acf0-4239aec34986 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:01.138094 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d7d8fe-9aeb-4875-b0ca-fdb03ec447a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:01.138846 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7d8fe-9aeb-4875-b0ca-fdb03ec447a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:01.139169 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7d8fe-9aeb-4875-b0ca-fdb03ec447a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:01.139489 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7d8fe-9aeb-4875-b0ca-fdb03ec447a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:01.140301 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d7d8fe-9aeb-4875-b0ca-fdb03ec447a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:47:01.140661 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d7d8fe-9aeb-4875-b0ca-fdb03ec447a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:01.142573 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d7d8fe-9aeb-4875-b0ca-fdb03ec447a0 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:01.157389 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1437/2834] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:47:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:47:01.162630 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87261841-e9a1-4533-977e-d24e5c87e2f3 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:01.173258 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87261841-e9a1-4533-977e-d24e5c87e2f3 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:01.179451 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74031d95-4586-4c33-9961-c37a4a3a1097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:01.180285 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74031d95-4586-4c33-9961-c37a4a3a1097 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:01.181209 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74031d95-4586-4c33-9961-c37a4a3a1097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:01.181209 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74031d95-4586-4c33-9961-c37a4a3a1097 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:01.182304 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74031d95-4586-4c33-9961-c37a4a3a1097 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:01.184380 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74031d95-4586-4c33-9961-c37a4a3a1097 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:01.217817 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1398/2835] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:47:01 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:47:01.222526 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d7d8fe-9aeb-4875-b0ca-fdb03ec447a0 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:01.234689 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d7d8fe-9aeb-4875-b0ca-fdb03ec447a0 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:01.272685 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:47:01.308938 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93b170c-7995-41c6-97d8-d5c9a3f239cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:01.309825 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93b170c-7995-41c6-97d8-d5c9a3f239cb admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:01.310151 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93b170c-7995-41c6-97d8-d5c9a3f239cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:01.310464 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93b170c-7995-41c6-97d8-d5c9a3f239cb admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:01.311214 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93b170c-7995-41c6-97d8-d5c9a3f239cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:01.313578 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93b170c-7995-41c6-97d8-d5c9a3f239cb admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:01.346707 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1438/2836] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:47:01 2022] POST /identity/v3/users => generated 289 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:47:01.352059 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74031d95-4586-4c33-9961-c37a4a3a1097 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:01.359576 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:47:01.366376 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74031d95-4586-4c33-9961-c37a4a3a1097 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:01.374531 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a0fed-cbf8-4922-b009-9ddc6371d3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:01.375491 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a0fed-cbf8-4922-b009-9ddc6371d3f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:01.376084 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a0fed-cbf8-4922-b009-9ddc6371d3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:01.376513 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a0fed-cbf8-4922-b009-9ddc6371d3f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:01.377556 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498a0fed-cbf8-4922-b009-9ddc6371d3f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:01.379938 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498a0fed-cbf8-4922-b009-9ddc6371d3f5 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:01.394024 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1399/2837] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:47:01 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:47:01.398945 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93b170c-7995-41c6-97d8-d5c9a3f239cb admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:01.410455 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93b170c-7995-41c6-97d8-d5c9a3f239cb admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:01.416559 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee91485-0468-4d87-bdb8-bee8fdba88aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:01.417255 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee91485-0468-4d87-bdb8-bee8fdba88aa admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:01.417541 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee91485-0468-4d87-bdb8-bee8fdba88aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:01.417850 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee91485-0468-4d87-bdb8-bee8fdba88aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:01.418588 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee91485-0468-4d87-bdb8-bee8fdba88aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:01.420241 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee91485-0468-4d87-bdb8-bee8fdba88aa admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:01.433277 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1439/2838] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:47:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:47:01.438047 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a0fed-cbf8-4922-b009-9ddc6371d3f5 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:01.449888 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a0fed-cbf8-4922-b009-9ddc6371d3f5 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:01.455391 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e567916-3f73-4a42-a993-682ab20d2b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:01.456380 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e567916-3f73-4a42-a993-682ab20d2b67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:01.456700 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e567916-3f73-4a42-a993-682ab20d2b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:01.457365 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e567916-3f73-4a42-a993-682ab20d2b67 admin admin] PATH_INFO: `/v3/projects/0feed75be10a4da5b948abee0f0b4d1d/users/3fb36fbb26594516892d4a03e0bf14d9/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:01.476933 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e567916-3f73-4a42-a993-682ab20d2b67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0feed75be10a4da5b948abee0f0b4d1d, user_id=3fb36fbb26594516892d4a03e0bf14d9, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:01.480254 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e567916-3f73-4a42-a993-682ab20d2b67 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:01.501985 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1400/2839] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:47:01 2022] PUT /identity/v3/projects/0feed75be10a4da5b948abee0f0b4d1d/users/3fb36fbb26594516892d4a03e0bf14d9/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:47:01.514856 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31766ec7-f207-4b44-ab73-f85f19bd6893 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:01.515364 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31766ec7-f207-4b44-ab73-f85f19bd6893 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:01.515906 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31766ec7-f207-4b44-ab73-f85f19bd6893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:01.553943 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-31766ec7-f207-4b44-ab73-f85f19bd6893 None None] MFA Rules not processed for user `3fb36fbb26594516892d4a03e0bf14d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:47:01.555209 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-31766ec7-f207-4b44-ab73-f85f19bd6893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:47:01.652179 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1440/2840] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:47:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:47:01.664696 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e567916-3f73-4a42-a993-682ab20d2b67 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:01.667403 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0e567916-3f73-4a42-a993-682ab20d2b67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:47:01.740472 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e567916-3f73-4a42-a993-682ab20d2b67 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:01.745326 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d120ca9-fc7f-4fb1-9812-ff21c94c7afc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7da4bf6f3be47b4aa25c6240cb4be03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:01.746211 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d120ca9-fc7f-4fb1-9812-ff21c94c7afc service glance] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:01.746608 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d120ca9-fc7f-4fb1-9812-ff21c94c7afc service glance] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:01.746987 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d120ca9-fc7f-4fb1-9812-ff21c94c7afc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:01.758133 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d120ca9-fc7f-4fb1-9812-ff21c94c7afc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:01.759712 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d120ca9-fc7f-4fb1-9812-ff21c94c7afc service glance] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:01.770725 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1401/2841] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:47:01 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:47:11.648834 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=51,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:47:11.766013 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=39,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:47:28.415866 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31766ec7-f207-4b44-ab73-f85f19bd6893 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:28.501867 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31766ec7-f207-4b44-ab73-f85f19bd6893 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:28.507340 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c81e88-49fd-4506-b7e1-5f8e575c156f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:28.508645 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c81e88-49fd-4506-b7e1-5f8e575c156f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:28.508931 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c81e88-49fd-4506-b7e1-5f8e575c156f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:28.509406 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c81e88-49fd-4506-b7e1-5f8e575c156f admin admin] PATH_INFO: `/v3/users/3fb36fbb26594516892d4a03e0bf14d9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:28.510887 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c81e88-49fd-4506-b7e1-5f8e575c156f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fb36fbb26594516892d4a03e0bf14d9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:28.512909 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c81e88-49fd-4506-b7e1-5f8e575c156f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:28.574206 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c2c81e88-49fd-4506-b7e1-5f8e575c156f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fb36fbb26594516892d4a03e0bf14d9'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:28.593099 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c2c81e88-49fd-4506-b7e1-5f8e575c156f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fb36fbb26594516892d4a03e0bf14d9'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:28.594182 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c2c81e88-49fd-4506-b7e1-5f8e575c156f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fb36fbb26594516892d4a03e0bf14d9'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:28.595059 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c2c81e88-49fd-4506-b7e1-5f8e575c156f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fb36fbb26594516892d4a03e0bf14d9'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:28.605799 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c2c81e88-49fd-4506-b7e1-5f8e575c156f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fb36fbb26594516892d4a03e0bf14d9'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:28.623406 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1441/2842] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:47:28 2022] DELETE /identity/v3/users/3fb36fbb26594516892d4a03e0bf14d9 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:47:28.627942 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec6b474-c300-4b83-bf0d-ffc5b179e8fa None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:28.628533 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec6b474-c300-4b83-bf0d-ffc5b179e8fa None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:28.628660 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec6b474-c300-4b83-bf0d-ffc5b179e8fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:28.650096 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3ec6b474-c300-4b83-bf0d-ffc5b179e8fa None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:47:28.651241 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3ec6b474-c300-4b83-bf0d-ffc5b179e8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:47:28.741717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1402/2843] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:47:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:47:28.752130 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c81e88-49fd-4506-b7e1-5f8e575c156f admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:28.753674 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c2c81e88-49fd-4506-b7e1-5f8e575c156f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:47:28.823074 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c81e88-49fd-4506-b7e1-5f8e575c156f admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:28.827182 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced8118-8a0d-4ac5-8a98-922f9846053c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:28.827897 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced8118-8a0d-4ac5-8a98-922f9846053c service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:28.828186 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced8118-8a0d-4ac5-8a98-922f9846053c service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:28.828464 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced8118-8a0d-4ac5-8a98-922f9846053c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:28.840589 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dced8118-8a0d-4ac5-8a98-922f9846053c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:28.841817 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dced8118-8a0d-4ac5-8a98-922f9846053c service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:28.853386 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1442/2844] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:47:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:47:29.179436 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec6b474-c300-4b83-bf0d-ffc5b179e8fa None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:29.181870 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3ec6b474-c300-4b83-bf0d-ffc5b179e8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:47:29.204241 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec6b474-c300-4b83-bf0d-ffc5b179e8fa None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:29.213080 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d4287-f42b-46a3-af0a-dd87017b9baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:29.214506 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d4287-f42b-46a3-af0a-dd87017b9baf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:29.214879 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d4287-f42b-46a3-af0a-dd87017b9baf admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:29.215302 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d4287-f42b-46a3-af0a-dd87017b9baf admin admin] PATH_INFO: `/v3/projects/0feed75be10a4da5b948abee0f0b4d1d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:29.217305 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41d4287-f42b-46a3-af0a-dd87017b9baf admin admin] RBAC: Authorizing `identity:delete_project(project_id=0feed75be10a4da5b948abee0f0b4d1d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:29.219829 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41d4287-f42b-46a3-af0a-dd87017b9baf admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:29.290214 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b41d4287-f42b-46a3-af0a-dd87017b9baf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0feed75be10a4da5b948abee0f0b4d1d', 'request_id': 'req-b41d4287-f42b-46a3-af0a-dd87017b9baf'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:29.299689 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b41d4287-f42b-46a3-af0a-dd87017b9baf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0feed75be10a4da5b948abee0f0b4d1d', 'request_id': 'req-b41d4287-f42b-46a3-af0a-dd87017b9baf'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:29.306882 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b41d4287-f42b-46a3-af0a-dd87017b9baf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0feed75be10a4da5b948abee0f0b4d1d', 'request_id': 'req-b41d4287-f42b-46a3-af0a-dd87017b9baf'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:29.320486 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b41d4287-f42b-46a3-af0a-dd87017b9baf admin admin] The token cache is being invalidate because project 0feed75be10a4da5b948abee0f0b4d1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:47:29.320822 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b41d4287-f42b-46a3-af0a-dd87017b9baf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:29.321562 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b41d4287-f42b-46a3-af0a-dd87017b9baf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:29.323335 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1403/2845] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:47:29 2022] DELETE /identity/v3/projects/0feed75be10a4da5b948abee0f0b4d1d => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:47:29.328710 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3cc71-ba94-40a0-8f7c-aa09cdeb5df4 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:29.329104 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3cc71-ba94-40a0-8f7c-aa09cdeb5df4 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:29.329474 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3cc71-ba94-40a0-8f7c-aa09cdeb5df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:29.350883 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-f2b3cc71-ba94-40a0-8f7c-aa09cdeb5df4 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:47:29.352418 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f2b3cc71-ba94-40a0-8f7c-aa09cdeb5df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:47:29.376151 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:47:29.429013 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1443/2846] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:47:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:47:29.437486 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abab309-34ae-4185-8a77-f9850cb3b3b1 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:29.437843 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abab309-34ae-4185-8a77-f9850cb3b3b1 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:29.438126 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abab309-34ae-4185-8a77-f9850cb3b3b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:29.459482 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-2abab309-34ae-4185-8a77-f9850cb3b3b1 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:47:29.460732 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2abab309-34ae-4185-8a77-f9850cb3b3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:47:29.470152 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1404/2847] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:47:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:47:29.474912 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b3cc71-ba94-40a0-8f7c-aa09cdeb5df4 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:29.485851 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b3cc71-ba94-40a0-8f7c-aa09cdeb5df4 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:29.491267 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32167c16-d129-4516-b26c-6f61ee8fd8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:29.491936 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32167c16-d129-4516-b26c-6f61ee8fd8a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:29.492209 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32167c16-d129-4516-b26c-6f61ee8fd8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:29.492488 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32167c16-d129-4516-b26c-6f61ee8fd8a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:29.493236 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32167c16-d129-4516-b26c-6f61ee8fd8a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:47:29.493546 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32167c16-d129-4516-b26c-6f61ee8fd8a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:29.495253 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32167c16-d129-4516-b26c-6f61ee8fd8a8 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:29.508802 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1444/2848] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:47:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:47:29.514976 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abab309-34ae-4185-8a77-f9850cb3b3b1 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:29.529331 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abab309-34ae-4185-8a77-f9850cb3b3b1 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:29.535456 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aac9c6-fd29-465c-82d7-0c0c0ec67f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:29.536011 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aac9c6-fd29-465c-82d7-0c0c0ec67f08 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:29.536399 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aac9c6-fd29-465c-82d7-0c0c0ec67f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:29.536780 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aac9c6-fd29-465c-82d7-0c0c0ec67f08 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:29.537500 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aac9c6-fd29-465c-82d7-0c0c0ec67f08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:29.539481 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aac9c6-fd29-465c-82d7-0c0c0ec67f08 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:29.573131 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1405/2849] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:47:29 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:47:29.577437 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32167c16-d129-4516-b26c-6f61ee8fd8a8 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:29.589352 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32167c16-d129-4516-b26c-6f61ee8fd8a8 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:29.660152 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61acffc0-1103-43f2-8db6-929692c75180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:29.661101 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61acffc0-1103-43f2-8db6-929692c75180 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:29.661224 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61acffc0-1103-43f2-8db6-929692c75180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:29.661459 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61acffc0-1103-43f2-8db6-929692c75180 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:29.662286 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61acffc0-1103-43f2-8db6-929692c75180 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:29.664379 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61acffc0-1103-43f2-8db6-929692c75180 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:29.686419 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1445/2850] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:47:29 2022] POST /identity/v3/users => generated 292 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:47:29.691157 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aac9c6-fd29-465c-82d7-0c0c0ec67f08 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:29.703623 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aac9c6-fd29-465c-82d7-0c0c0ec67f08 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:29.709428 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8332094-b1ec-4b4f-ab10-f40da7408a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:29.710204 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8332094-b1ec-4b4f-ab10-f40da7408a2a admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:29.710551 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8332094-b1ec-4b4f-ab10-f40da7408a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:29.710876 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8332094-b1ec-4b4f-ab10-f40da7408a2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:29.711664 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8332094-b1ec-4b4f-ab10-f40da7408a2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:29.713529 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8332094-b1ec-4b4f-ab10-f40da7408a2a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:29.728726 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1406/2851] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:47:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:47:29.736539 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61acffc0-1103-43f2-8db6-929692c75180 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:29.752229 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61acffc0-1103-43f2-8db6-929692c75180 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:29.758179 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfbef9c-413d-4e62-b603-5a6d01346717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:29.759550 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfbef9c-413d-4e62-b603-5a6d01346717 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:29.759820 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfbef9c-413d-4e62-b603-5a6d01346717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:29.760164 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfbef9c-413d-4e62-b603-5a6d01346717 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:29.761105 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfbef9c-413d-4e62-b603-5a6d01346717 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:29.762955 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfbef9c-413d-4e62-b603-5a6d01346717 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:29.775970 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1446/2852] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:47:29 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:47:29.780764 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8332094-b1ec-4b4f-ab10-f40da7408a2a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:29.792514 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8332094-b1ec-4b4f-ab10-f40da7408a2a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:29.797603 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae1823e-9f8a-435f-9498-7a68924e4110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:29.798586 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1823e-9f8a-435f-9498-7a68924e4110 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:29.798862 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1823e-9f8a-435f-9498-7a68924e4110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:29.799147 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1823e-9f8a-435f-9498-7a68924e4110 admin admin] PATH_INFO: `/v3/projects/12ffe82d4c884937ba5ddc0eebb19a0d/users/7f4ab1b045e24489ab726b2d0305af98/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:29.819859 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae1823e-9f8a-435f-9498-7a68924e4110 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12ffe82d4c884937ba5ddc0eebb19a0d, user_id=7f4ab1b045e24489ab726b2d0305af98, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:29.822981 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae1823e-9f8a-435f-9498-7a68924e4110 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:29.843322 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1407/2853] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:47:29 2022] PUT /identity/v3/projects/12ffe82d4c884937ba5ddc0eebb19a0d/users/7f4ab1b045e24489ab726b2d0305af98/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:47:29.856728 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe1fe83-a493-42e0-a3dc-697c3ab1e2ae None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:29.857121 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe1fe83-a493-42e0-a3dc-697c3ab1e2ae None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:29.857449 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe1fe83-a493-42e0-a3dc-697c3ab1e2ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:29.890974 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-ffe1fe83-a493-42e0-a3dc-697c3ab1e2ae None None] MFA Rules not processed for user `7f4ab1b045e24489ab726b2d0305af98`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:47:29.892214 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ffe1fe83-a493-42e0-a3dc-697c3ab1e2ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:47:29.974964 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1447/2854] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:47:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:47:29.980647 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae1823e-9f8a-435f-9498-7a68924e4110 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:30.006273 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:47:30.064024 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae1823e-9f8a-435f-9498-7a68924e4110 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:30.079079 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a80aef-050b-4dd8-8981-1b4a9f6662d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:30.079780 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a80aef-050b-4dd8-8981-1b4a9f6662d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:30.080071 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a80aef-050b-4dd8-8981-1b4a9f6662d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:30.080353 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a80aef-050b-4dd8-8981-1b4a9f6662d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:30.081051 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a80aef-050b-4dd8-8981-1b4a9f6662d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:30.083451 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a80aef-050b-4dd8-8981-1b4a9f6662d1 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:30.117505 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1408/2855] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:47:29 2022] POST /identity/v3/projects => generated 385 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:47:30.122659 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe1fe83-a493-42e0-a3dc-697c3ab1e2ae None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:30.135154 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe1fe83-a493-42e0-a3dc-697c3ab1e2ae None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:30.149393 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:47:30.199132 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dca9cd-2e4c-452b-830d-7887b80f7091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:30.199820 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dca9cd-2e4c-452b-830d-7887b80f7091 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:30.200111 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dca9cd-2e4c-452b-830d-7887b80f7091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:30.200392 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dca9cd-2e4c-452b-830d-7887b80f7091 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:30.201103 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dca9cd-2e4c-452b-830d-7887b80f7091 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:30.203120 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dca9cd-2e4c-452b-830d-7887b80f7091 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:30.227814 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1448/2856] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:47:30 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:47:30.233941 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a80aef-050b-4dd8-8981-1b4a9f6662d1 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:30.245979 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a80aef-050b-4dd8-8981-1b4a9f6662d1 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:30.251294 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3ccd49-da39-4ef0-9eac-3de75968f3fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:30.251990 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3ccd49-da39-4ef0-9eac-3de75968f3fd admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:30.252428 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3ccd49-da39-4ef0-9eac-3de75968f3fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:30.252720 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3ccd49-da39-4ef0-9eac-3de75968f3fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:30.253461 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3ccd49-da39-4ef0-9eac-3de75968f3fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:30.255193 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3ccd49-da39-4ef0-9eac-3de75968f3fd admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:30.272214 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1409/2857] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:47:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:47:30.275320 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dca9cd-2e4c-452b-830d-7887b80f7091 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:30.287293 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dca9cd-2e4c-452b-830d-7887b80f7091 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:30.293292 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1959e806-9777-4279-a25b-904cd9ff04da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:30.294031 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959e806-9777-4279-a25b-904cd9ff04da admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:30.294321 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959e806-9777-4279-a25b-904cd9ff04da admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:30.294602 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959e806-9777-4279-a25b-904cd9ff04da admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:30.295349 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1959e806-9777-4279-a25b-904cd9ff04da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:30.297011 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1959e806-9777-4279-a25b-904cd9ff04da admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:30.308762 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1449/2858] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:47:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:47:30.313484 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3ccd49-da39-4ef0-9eac-3de75968f3fd admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:30.324690 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3ccd49-da39-4ef0-9eac-3de75968f3fd admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:30.330022 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b9e221-8513-4b03-91b2-96423276e011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:30.331074 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9e221-8513-4b03-91b2-96423276e011 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:30.331438 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9e221-8513-4b03-91b2-96423276e011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:30.332069 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9e221-8513-4b03-91b2-96423276e011 admin admin] PATH_INFO: `/v3/projects/630174f46aaa4945958d57d1bad039b3/users/c274d0c0965e48f09114c30f153e532b/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:30.350859 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b9e221-8513-4b03-91b2-96423276e011 admin admin] RBAC: Authorizing `identity:create_grant(project_id=630174f46aaa4945958d57d1bad039b3, user_id=c274d0c0965e48f09114c30f153e532b, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:30.353831 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b9e221-8513-4b03-91b2-96423276e011 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:30.370866 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1410/2859] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:47:30 2022] PUT /identity/v3/projects/630174f46aaa4945958d57d1bad039b3/users/c274d0c0965e48f09114c30f153e532b/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:47:30.383281 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d54d45-6e7c-4874-a68c-9c7b1f1a59a3 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:30.383728 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d54d45-6e7c-4874-a68c-9c7b1f1a59a3 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:30.384109 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d54d45-6e7c-4874-a68c-9c7b1f1a59a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:30.419888 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-53d54d45-6e7c-4874-a68c-9c7b1f1a59a3 None None] MFA Rules not processed for user `c274d0c0965e48f09114c30f153e532b`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:47:30.421808 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-53d54d45-6e7c-4874-a68c-9c7b1f1a59a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:47:30.534356 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1450/2860] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:47:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:47:30.540224 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b9e221-8513-4b03-91b2-96423276e011 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:30.629539 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b9e221-8513-4b03-91b2-96423276e011 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:30.635264 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f868c-20e3-4531-8f5c-653152c6bb87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:30.635965 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f868c-20e3-4531-8f5c-653152c6bb87 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:30.636255 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f868c-20e3-4531-8f5c-653152c6bb87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:30.636537 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f868c-20e3-4531-8f5c-653152c6bb87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:30.637250 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161f868c-20e3-4531-8f5c-653152c6bb87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:30.639242 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161f868c-20e3-4531-8f5c-653152c6bb87 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:30.670390 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1411/2861] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:47:30 2022] POST /identity/v3/projects => generated 385 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:47:30.674957 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d54d45-6e7c-4874-a68c-9c7b1f1a59a3 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:30.687100 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d54d45-6e7c-4874-a68c-9c7b1f1a59a3 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:30.766731 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370adbe5-08f9-4d3f-b672-e2fe9f7021bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:30.767417 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370adbe5-08f9-4d3f-b672-e2fe9f7021bf admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:30.767714 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370adbe5-08f9-4d3f-b672-e2fe9f7021bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:30.767996 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370adbe5-08f9-4d3f-b672-e2fe9f7021bf admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:30.768739 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370adbe5-08f9-4d3f-b672-e2fe9f7021bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:30.771101 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370adbe5-08f9-4d3f-b672-e2fe9f7021bf admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:30.800876 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1451/2862] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:47:30 2022] POST /identity/v3/users => generated 299 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:47:30.806183 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f868c-20e3-4531-8f5c-653152c6bb87 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:30.825904 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f868c-20e3-4531-8f5c-653152c6bb87 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:30.835099 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb6feb5-9141-4090-a3d8-fc9aaa93c70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:30.836275 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6feb5-9141-4090-a3d8-fc9aaa93c70f admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:30.836762 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6feb5-9141-4090-a3d8-fc9aaa93c70f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:30.837255 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6feb5-9141-4090-a3d8-fc9aaa93c70f admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:30.838667 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb6feb5-9141-4090-a3d8-fc9aaa93c70f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:30.841776 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb6feb5-9141-4090-a3d8-fc9aaa93c70f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:30.860920 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1412/2863] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:47:30 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:47:30.875194 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370adbe5-08f9-4d3f-b672-e2fe9f7021bf admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:30.891010 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370adbe5-08f9-4d3f-b672-e2fe9f7021bf admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:30.896994 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101412d4-ea2e-41cb-b0bb-192409bcfdbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:30.898083 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101412d4-ea2e-41cb-b0bb-192409bcfdbc admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:30.898543 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101412d4-ea2e-41cb-b0bb-192409bcfdbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:30.899038 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101412d4-ea2e-41cb-b0bb-192409bcfdbc admin admin] PATH_INFO: `/v3/projects/5d328a4896f04dd1b489c2e95bf9a56d/users/9eadfd4390e64280808c7fc16d429545/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:30.913973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:47:30.919677 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101412d4-ea2e-41cb-b0bb-192409bcfdbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d328a4896f04dd1b489c2e95bf9a56d, user_id=9eadfd4390e64280808c7fc16d429545, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:30.923190 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101412d4-ea2e-41cb-b0bb-192409bcfdbc admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:30.948651 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1452/2864] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:47:30 2022] PUT /identity/v3/projects/5d328a4896f04dd1b489c2e95bf9a56d/users/9eadfd4390e64280808c7fc16d429545/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:47:30.954531 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb6feb5-9141-4090-a3d8-fc9aaa93c70f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:30.970605 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb6feb5-9141-4090-a3d8-fc9aaa93c70f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:31.040099 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:47:31.061829 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c784522-4777-44a3-8534-1ffe3ca66b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:31.062574 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c784522-4777-44a3-8534-1ffe3ca66b5a admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:31.062913 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c784522-4777-44a3-8534-1ffe3ca66b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:31.063258 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c784522-4777-44a3-8534-1ffe3ca66b5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:31.064255 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c784522-4777-44a3-8534-1ffe3ca66b5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:31.066113 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c784522-4777-44a3-8534-1ffe3ca66b5a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:31.084910 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1413/2865] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:47:30 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:47:31.091960 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101412d4-ea2e-41cb-b0bb-192409bcfdbc admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:31.112445 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101412d4-ea2e-41cb-b0bb-192409bcfdbc admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:31.120248 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7af0cf-c671-4ba2-9b3c-b5d8ba15e418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:31.122169 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7af0cf-c671-4ba2-9b3c-b5d8ba15e418 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:31.122396 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7af0cf-c671-4ba2-9b3c-b5d8ba15e418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:31.122862 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7af0cf-c671-4ba2-9b3c-b5d8ba15e418 admin admin] PATH_INFO: `/v3/projects/5d328a4896f04dd1b489c2e95bf9a56d/users/9eadfd4390e64280808c7fc16d429545/roles/98f0943a610f4a9e9f8de54f6e9f717d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:31.126483 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7af0cf-c671-4ba2-9b3c-b5d8ba15e418 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d328a4896f04dd1b489c2e95bf9a56d, user_id=9eadfd4390e64280808c7fc16d429545, role_id=98f0943a610f4a9e9f8de54f6e9f717d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:31.131304 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7af0cf-c671-4ba2-9b3c-b5d8ba15e418 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:31.152091 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1453/2866] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:47:31 2022] PUT /identity/v3/projects/5d328a4896f04dd1b489c2e95bf9a56d/users/9eadfd4390e64280808c7fc16d429545/roles/98f0943a610f4a9e9f8de54f6e9f717d => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:47:31.166384 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc98ac-678d-47c0-9b73-9bf5da7eb2ff None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:31.166754 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc98ac-678d-47c0-9b73-9bf5da7eb2ff None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:31.167081 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc98ac-678d-47c0-9b73-9bf5da7eb2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:31.199674 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-41fc98ac-678d-47c0-9b73-9bf5da7eb2ff None None] MFA Rules not processed for user `9eadfd4390e64280808c7fc16d429545`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:47:31.200966 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-41fc98ac-678d-47c0-9b73-9bf5da7eb2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:47:31.303484 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1414/2867] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:47:31 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:47:31.317249 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7af0cf-c671-4ba2-9b3c-b5d8ba15e418 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:31.321386 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3d7af0cf-c671-4ba2-9b3c-b5d8ba15e418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:47:31.423827 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7af0cf-c671-4ba2-9b3c-b5d8ba15e418 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:31.431247 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f462428-23f7-4e06-9e55-4096c9ed7363 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:31.432390 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f462428-23f7-4e06-9e55-4096c9ed7363 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:31.432884 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f462428-23f7-4e06-9e55-4096c9ed7363 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:31.433391 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f462428-23f7-4e06-9e55-4096c9ed7363 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:31.527838 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f462428-23f7-4e06-9e55-4096c9ed7363 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:31.529333 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f462428-23f7-4e06-9e55-4096c9ed7363 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:31.541957 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1454/2868] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:47:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:47:33.287978 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc98ac-678d-47c0-9b73-9bf5da7eb2ff None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:33.290288 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-41fc98ac-678d-47c0-9b73-9bf5da7eb2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:47:33.304447 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc98ac-678d-47c0-9b73-9bf5da7eb2ff None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:33.309163 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1a39cd-16a3-4709-80c1-529f31b88388 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:33.309888 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a39cd-16a3-4709-80c1-529f31b88388 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:33.310184 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a39cd-16a3-4709-80c1-529f31b88388 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:33.310481 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a39cd-16a3-4709-80c1-529f31b88388 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:33.381615 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1a39cd-16a3-4709-80c1-529f31b88388 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:33.383108 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1a39cd-16a3-4709-80c1-529f31b88388 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:33.395349 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1415/2869] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:47:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:47:34.118381 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f462428-23f7-4e06-9e55-4096c9ed7363 service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:34.130383 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f462428-23f7-4e06-9e55-4096c9ed7363 service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:34.135794 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527e20e-5aaa-4379-8f39-9bf5d7dddbde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:34.136476 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527e20e-5aaa-4379-8f39-9bf5d7dddbde service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:34.136764 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527e20e-5aaa-4379-8f39-9bf5d7dddbde service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:34.137037 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527e20e-5aaa-4379-8f39-9bf5d7dddbde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:34.148618 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527e20e-5aaa-4379-8f39-9bf5d7dddbde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:34.150147 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527e20e-5aaa-4379-8f39-9bf5d7dddbde service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:34.163696 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1455/2870] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:47:34 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:47:38.210120 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1a39cd-16a3-4709-80c1-529f31b88388 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:38.296179 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1a39cd-16a3-4709-80c1-529f31b88388 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:38.302734 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455e241b-cd87-4fd5-b2e1-96a460052ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:38.303729 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e241b-cd87-4fd5-b2e1-96a460052ed8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:38.304063 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e241b-cd87-4fd5-b2e1-96a460052ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:38.304417 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e241b-cd87-4fd5-b2e1-96a460052ed8 admin admin] PATH_INFO: `/v3/users/7f4ab1b045e24489ab726b2d0305af98` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:38.306004 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455e241b-cd87-4fd5-b2e1-96a460052ed8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f4ab1b045e24489ab726b2d0305af98)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:38.308056 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455e241b-cd87-4fd5-b2e1-96a460052ed8 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:38.314444 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:47:38.385197 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-455e241b-cd87-4fd5-b2e1-96a460052ed8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f4ab1b045e24489ab726b2d0305af98'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:38.398338 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-455e241b-cd87-4fd5-b2e1-96a460052ed8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f4ab1b045e24489ab726b2d0305af98'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:38.399747 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-455e241b-cd87-4fd5-b2e1-96a460052ed8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f4ab1b045e24489ab726b2d0305af98'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:38.400613 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-455e241b-cd87-4fd5-b2e1-96a460052ed8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f4ab1b045e24489ab726b2d0305af98'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:38.422545 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-455e241b-cd87-4fd5-b2e1-96a460052ed8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f4ab1b045e24489ab726b2d0305af98'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:38.441002 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1416/2871] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:47:38 2022] DELETE /identity/v3/users/7f4ab1b045e24489ab726b2d0305af98 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:47:38.446583 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c9654-0b56-4a91-93ed-e8f5cec7ac15 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:38.447348 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c9654-0b56-4a91-93ed-e8f5cec7ac15 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:38.447348 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c9654-0b56-4a91-93ed-e8f5cec7ac15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:38.466236 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-a59c9654-0b56-4a91-93ed-e8f5cec7ac15 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:47:38.467507 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a59c9654-0b56-4a91-93ed-e8f5cec7ac15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:47:38.550795 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1456/2872] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:47:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:47:38.564796 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455e241b-cd87-4fd5-b2e1-96a460052ed8 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:38.565962 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-455e241b-cd87-4fd5-b2e1-96a460052ed8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:47:38.681194 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455e241b-cd87-4fd5-b2e1-96a460052ed8 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:38.687557 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e1c85-7c45-4254-bc74-494f57bdc830 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:38.688721 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e1c85-7c45-4254-bc74-494f57bdc830 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:38.689253 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e1c85-7c45-4254-bc74-494f57bdc830 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:38.689821 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e1c85-7c45-4254-bc74-494f57bdc830 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:38.708715 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917e1c85-7c45-4254-bc74-494f57bdc830 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:38.711560 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917e1c85-7c45-4254-bc74-494f57bdc830 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:38.731599 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1417/2873] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:47:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:47:39.063643 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59c9654-0b56-4a91-93ed-e8f5cec7ac15 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:39.065591 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a59c9654-0b56-4a91-93ed-e8f5cec7ac15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:47:39.080564 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59c9654-0b56-4a91-93ed-e8f5cec7ac15 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:39.093484 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1f2717-140d-4298-b495-456c69a9ce62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:39.095571 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f2717-140d-4298-b495-456c69a9ce62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:39.096073 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f2717-140d-4298-b495-456c69a9ce62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:39.096543 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f2717-140d-4298-b495-456c69a9ce62 admin admin] PATH_INFO: `/v3/projects/12ffe82d4c884937ba5ddc0eebb19a0d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:39.099301 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1f2717-140d-4298-b495-456c69a9ce62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12ffe82d4c884937ba5ddc0eebb19a0d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:39.103088 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1f2717-140d-4298-b495-456c69a9ce62 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:39.135392 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:47:39.172660 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8c1f2717-140d-4298-b495-456c69a9ce62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12ffe82d4c884937ba5ddc0eebb19a0d', 'request_id': 'req-8c1f2717-140d-4298-b495-456c69a9ce62'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:39.179211 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8c1f2717-140d-4298-b495-456c69a9ce62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12ffe82d4c884937ba5ddc0eebb19a0d', 'request_id': 'req-8c1f2717-140d-4298-b495-456c69a9ce62'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:39.191801 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8c1f2717-140d-4298-b495-456c69a9ce62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12ffe82d4c884937ba5ddc0eebb19a0d', 'request_id': 'req-8c1f2717-140d-4298-b495-456c69a9ce62'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:39.206390 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8c1f2717-140d-4298-b495-456c69a9ce62 admin admin] The token cache is being invalidate because project 12ffe82d4c884937ba5ddc0eebb19a0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:47:39.207200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8c1f2717-140d-4298-b495-456c69a9ce62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:39.208172 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8c1f2717-140d-4298-b495-456c69a9ce62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:39.210090 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1457/2874] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:47:39 2022] DELETE /identity/v3/projects/12ffe82d4c884937ba5ddc0eebb19a0d => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:47:39.214859 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e1c85-7c45-4254-bc74-494f57bdc830 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:39.230755 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e1c85-7c45-4254-bc74-494f57bdc830 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:39.313297 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce42f6-b186-49b3-8085-5ac0967b1604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:39.314247 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce42f6-b186-49b3-8085-5ac0967b1604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:39.314535 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce42f6-b186-49b3-8085-5ac0967b1604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:39.314824 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce42f6-b186-49b3-8085-5ac0967b1604 admin admin] PATH_INFO: `/v3/users/c274d0c0965e48f09114c30f153e532b` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:39.316601 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ce42f6-b186-49b3-8085-5ac0967b1604 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c274d0c0965e48f09114c30f153e532b)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:39.318736 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ce42f6-b186-49b3-8085-5ac0967b1604 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:39.355309 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:47:39.382299 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d5ce42f6-b186-49b3-8085-5ac0967b1604 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c274d0c0965e48f09114c30f153e532b'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:39.391144 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d5ce42f6-b186-49b3-8085-5ac0967b1604 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c274d0c0965e48f09114c30f153e532b'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:39.391905 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d5ce42f6-b186-49b3-8085-5ac0967b1604 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c274d0c0965e48f09114c30f153e532b'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:39.392526 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d5ce42f6-b186-49b3-8085-5ac0967b1604 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c274d0c0965e48f09114c30f153e532b'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:39.407827 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d5ce42f6-b186-49b3-8085-5ac0967b1604 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c274d0c0965e48f09114c30f153e532b'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:39.423630 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1418/2875] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:47:39 2022] DELETE /identity/v3/users/c274d0c0965e48f09114c30f153e532b => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:47:39.767313 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1f2717-140d-4298-b495-456c69a9ce62 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:39.768824 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8c1f2717-140d-4298-b495-456c69a9ce62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:47:39.782999 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1f2717-140d-4298-b495-456c69a9ce62 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:39.868020 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db46f34-0ca0-42aa-b06b-43351b6fd80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:39.868907 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db46f34-0ca0-42aa-b06b-43351b6fd80a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:39.869197 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db46f34-0ca0-42aa-b06b-43351b6fd80a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:39.869477 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db46f34-0ca0-42aa-b06b-43351b6fd80a admin admin] PATH_INFO: `/v3/projects/630174f46aaa4945958d57d1bad039b3` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:39.870770 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db46f34-0ca0-42aa-b06b-43351b6fd80a admin admin] RBAC: Authorizing `identity:delete_project(project_id=630174f46aaa4945958d57d1bad039b3)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:39.872762 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db46f34-0ca0-42aa-b06b-43351b6fd80a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:39.927210 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-2db46f34-0ca0-42aa-b06b-43351b6fd80a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '630174f46aaa4945958d57d1bad039b3', 'request_id': 'req-2db46f34-0ca0-42aa-b06b-43351b6fd80a'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:39.933941 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-2db46f34-0ca0-42aa-b06b-43351b6fd80a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '630174f46aaa4945958d57d1bad039b3', 'request_id': 'req-2db46f34-0ca0-42aa-b06b-43351b6fd80a'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:39.944961 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-2db46f34-0ca0-42aa-b06b-43351b6fd80a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '630174f46aaa4945958d57d1bad039b3', 'request_id': 'req-2db46f34-0ca0-42aa-b06b-43351b6fd80a'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:39.959469 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-2db46f34-0ca0-42aa-b06b-43351b6fd80a admin admin] The token cache is being invalidate because project 630174f46aaa4945958d57d1bad039b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:47:39.959787 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-2db46f34-0ca0-42aa-b06b-43351b6fd80a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:39.960697 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-2db46f34-0ca0-42aa-b06b-43351b6fd80a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:39.962551 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1458/2876] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:47:39 2022] DELETE /identity/v3/projects/630174f46aaa4945958d57d1bad039b3 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:47:39.967181 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce42f6-b186-49b3-8085-5ac0967b1604 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:39.968853 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d5ce42f6-b186-49b3-8085-5ac0967b1604 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:47:39.984581 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce42f6-b186-49b3-8085-5ac0967b1604 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:40.053179 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85af795-5105-468a-9a14-84af3f07e26b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:40.054404 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85af795-5105-468a-9a14-84af3f07e26b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:40.054553 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85af795-5105-468a-9a14-84af3f07e26b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:40.054739 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85af795-5105-468a-9a14-84af3f07e26b admin admin] PATH_INFO: `/v3/users/9eadfd4390e64280808c7fc16d429545` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:40.056053 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85af795-5105-468a-9a14-84af3f07e26b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9eadfd4390e64280808c7fc16d429545)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:40.058101 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85af795-5105-468a-9a14-84af3f07e26b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:40.120369 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a85af795-5105-468a-9a14-84af3f07e26b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9eadfd4390e64280808c7fc16d429545'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:40.128279 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a85af795-5105-468a-9a14-84af3f07e26b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9eadfd4390e64280808c7fc16d429545'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:40.129036 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a85af795-5105-468a-9a14-84af3f07e26b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9eadfd4390e64280808c7fc16d429545'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:40.129719 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a85af795-5105-468a-9a14-84af3f07e26b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9eadfd4390e64280808c7fc16d429545'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:40.146858 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a85af795-5105-468a-9a14-84af3f07e26b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9eadfd4390e64280808c7fc16d429545'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:40.164711 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1419/2877] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:47:39 2022] DELETE /identity/v3/users/9eadfd4390e64280808c7fc16d429545 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:47:40.535249 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db46f34-0ca0-42aa-b06b-43351b6fd80a admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:40.538607 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-2db46f34-0ca0-42aa-b06b-43351b6fd80a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:47:40.556249 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db46f34-0ca0-42aa-b06b-43351b6fd80a admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:40.601521 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:47:40.640468 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8765dc0d-a062-425b-a541-f0617beccfe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:40.641371 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765dc0d-a062-425b-a541-f0617beccfe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:40.641689 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765dc0d-a062-425b-a541-f0617beccfe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:40.641989 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765dc0d-a062-425b-a541-f0617beccfe9 admin admin] PATH_INFO: `/v3/projects/5d328a4896f04dd1b489c2e95bf9a56d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:40.649480 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8765dc0d-a062-425b-a541-f0617beccfe9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d328a4896f04dd1b489c2e95bf9a56d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:40.651769 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8765dc0d-a062-425b-a541-f0617beccfe9 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:40.716145 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8765dc0d-a062-425b-a541-f0617beccfe9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d328a4896f04dd1b489c2e95bf9a56d', 'request_id': 'req-8765dc0d-a062-425b-a541-f0617beccfe9'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:40.726806 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8765dc0d-a062-425b-a541-f0617beccfe9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d328a4896f04dd1b489c2e95bf9a56d', 'request_id': 'req-8765dc0d-a062-425b-a541-f0617beccfe9'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:40.738635 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8765dc0d-a062-425b-a541-f0617beccfe9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d328a4896f04dd1b489c2e95bf9a56d', 'request_id': 'req-8765dc0d-a062-425b-a541-f0617beccfe9'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:40.752973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8765dc0d-a062-425b-a541-f0617beccfe9 admin admin] The token cache is being invalidate because project 5d328a4896f04dd1b489c2e95bf9a56d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:47:40.753304 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8765dc0d-a062-425b-a541-f0617beccfe9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:40.753961 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8765dc0d-a062-425b-a541-f0617beccfe9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:40.755701 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1459/2878] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:47:40 2022] DELETE /identity/v3/projects/5d328a4896f04dd1b489c2e95bf9a56d => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:47:40.760941 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d09afe-b9a9-4f29-962c-bd161f764bc3 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:40.761863 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d09afe-b9a9-4f29-962c-bd161f764bc3 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:40.761863 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d09afe-b9a9-4f29-962c-bd161f764bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:40.782137 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a9d09afe-b9a9-4f29-962c-bd161f764bc3 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:47:40.783418 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a9d09afe-b9a9-4f29-962c-bd161f764bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:47:40.804822 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:47:40.858085 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1420/2879] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:47:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:47:40.866904 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f42bf-9f6c-45b2-94c3-facca8ee0108 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:40.867574 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f42bf-9f6c-45b2-94c3-facca8ee0108 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:40.867574 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f42bf-9f6c-45b2-94c3-facca8ee0108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:40.889152 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-ac4f42bf-9f6c-45b2-94c3-facca8ee0108 None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:47:40.890526 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ac4f42bf-9f6c-45b2-94c3-facca8ee0108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:47:40.905406 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1460/2880] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:47:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:47:40.911748 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d09afe-b9a9-4f29-962c-bd161f764bc3 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:40.922886 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d09afe-b9a9-4f29-962c-bd161f764bc3 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:40.928322 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3832dc8-6421-4788-87ae-a764d0491696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:40.929010 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3832dc8-6421-4788-87ae-a764d0491696 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:40.929295 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3832dc8-6421-4788-87ae-a764d0491696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:40.929571 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3832dc8-6421-4788-87ae-a764d0491696 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:40.930356 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3832dc8-6421-4788-87ae-a764d0491696 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:47:40.930795 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3832dc8-6421-4788-87ae-a764d0491696 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:40.932493 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3832dc8-6421-4788-87ae-a764d0491696 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:40.947192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1421/2881] 10.222.0.87 () {58 vars in 1152 bytes} [Tue Jun 21 18:47:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:47:40.952986 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4f42bf-9f6c-45b2-94c3-facca8ee0108 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:40.965284 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4f42bf-9f6c-45b2-94c3-facca8ee0108 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:40.970966 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e723cdc7-cfdb-44cf-98e3-c4392794717d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:40.971790 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e723cdc7-cfdb-44cf-98e3-c4392794717d admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:40.972167 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e723cdc7-cfdb-44cf-98e3-c4392794717d admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:40.972538 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e723cdc7-cfdb-44cf-98e3-c4392794717d admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:40.973931 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e723cdc7-cfdb-44cf-98e3-c4392794717d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:40.976010 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e723cdc7-cfdb-44cf-98e3-c4392794717d admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:41.008143 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1461/2882] 10.222.0.87 () {60 vars in 1152 bytes} [Tue Jun 21 18:47:40 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:47:41.012998 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3832dc8-6421-4788-87ae-a764d0491696 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:41.024104 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3832dc8-6421-4788-87ae-a764d0491696 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:41.107433 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d9f6c2-1005-4253-9ddb-b4633b3f2f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:41.108347 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9f6c2-1005-4253-9ddb-b4633b3f2f93 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:41.108671 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9f6c2-1005-4253-9ddb-b4633b3f2f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:41.108853 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9f6c2-1005-4253-9ddb-b4633b3f2f93 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:41.109533 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d9f6c2-1005-4253-9ddb-b4633b3f2f93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:41.111551 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d9f6c2-1005-4253-9ddb-b4633b3f2f93 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:41.151127 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1422/2883] 10.222.0.87 () {60 vars in 1143 bytes} [Tue Jun 21 18:47:41 2022] POST /identity/v3/users => generated 286 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:47:41.153481 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e723cdc7-cfdb-44cf-98e3-c4392794717d admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:41.171335 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e723cdc7-cfdb-44cf-98e3-c4392794717d admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:41.178449 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12c2902-30ed-4e49-bdf2-ccc830a161dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:41.179306 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c2902-30ed-4e49-bdf2-ccc830a161dd admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:41.179668 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c2902-30ed-4e49-bdf2-ccc830a161dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:41.180073 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c2902-30ed-4e49-bdf2-ccc830a161dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:41.181023 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12c2902-30ed-4e49-bdf2-ccc830a161dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:41.183240 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12c2902-30ed-4e49-bdf2-ccc830a161dd admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:41.201931 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1462/2884] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:47:41 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:47:41.207299 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d9f6c2-1005-4253-9ddb-b4633b3f2f93 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:41.221379 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d9f6c2-1005-4253-9ddb-b4633b3f2f93 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:41.226543 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098fa0c1-6505-4900-8a1f-5443f54e61a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:41.227239 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098fa0c1-6505-4900-8a1f-5443f54e61a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:41.227526 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098fa0c1-6505-4900-8a1f-5443f54e61a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:41.227804 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098fa0c1-6505-4900-8a1f-5443f54e61a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:41.228549 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098fa0c1-6505-4900-8a1f-5443f54e61a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:41.230227 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098fa0c1-6505-4900-8a1f-5443f54e61a2 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:41.241325 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1423/2885] 10.222.0.87 () {58 vars in 1121 bytes} [Tue Jun 21 18:47:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:47:41.247491 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12c2902-30ed-4e49-bdf2-ccc830a161dd admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:41.258739 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12c2902-30ed-4e49-bdf2-ccc830a161dd admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:41.263886 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaffb51-cf12-484b-8724-bca250517f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:41.264835 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaffb51-cf12-484b-8724-bca250517f91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:41.265119 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaffb51-cf12-484b-8724-bca250517f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:41.265401 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaffb51-cf12-484b-8724-bca250517f91 admin admin] PATH_INFO: `/v3/projects/551e24e8fae448ed9674fc7dfc38e880/users/e9cee90d8dfb418aaca1cee546d93068/roles/da6d4a0ab8c04809ba120170fb4ae8a9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:41.283669 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaffb51-cf12-484b-8724-bca250517f91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=551e24e8fae448ed9674fc7dfc38e880, user_id=e9cee90d8dfb418aaca1cee546d93068, role_id=da6d4a0ab8c04809ba120170fb4ae8a9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:41.286587 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaffb51-cf12-484b-8724-bca250517f91 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:41.303475 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1463/2886] 10.222.0.87 () {60 vars in 1482 bytes} [Tue Jun 21 18:47:41 2022] PUT /identity/v3/projects/551e24e8fae448ed9674fc7dfc38e880/users/e9cee90d8dfb418aaca1cee546d93068/roles/da6d4a0ab8c04809ba120170fb4ae8a9 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:47:41.376766 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683423d-70ae-4c00-9bc7-b07c60b70fe6 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:41.377232 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683423d-70ae-4c00-9bc7-b07c60b70fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:41.377757 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683423d-70ae-4c00-9bc7-b07c60b70fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:41.418079 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b683423d-70ae-4c00-9bc7-b07c60b70fe6 None None] MFA Rules not processed for user `e9cee90d8dfb418aaca1cee546d93068`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:47:41.419433 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b683423d-70ae-4c00-9bc7-b07c60b70fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:47:41.509515 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1424/2887] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:47:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:47:41.525440 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaffb51-cf12-484b-8724-bca250517f91 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:41.527757 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ecaffb51-cf12-484b-8724-bca250517f91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:47:41.625073 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaffb51-cf12-484b-8724-bca250517f91 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:41.630324 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad19ba7-0f33-4cb3-ab2f-3af8d9d5bd81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:41.630940 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad19ba7-0f33-4cb3-ab2f-3af8d9d5bd81 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:41.631228 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad19ba7-0f33-4cb3-ab2f-3af8d9d5bd81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:41.631509 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad19ba7-0f33-4cb3-ab2f-3af8d9d5bd81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:41.644655 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad19ba7-0f33-4cb3-ab2f-3af8d9d5bd81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:41.647131 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad19ba7-0f33-4cb3-ab2f-3af8d9d5bd81 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:41.659246 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1464/2888] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:47:41 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:47:41.690919 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b683423d-70ae-4c00-9bc7-b07c60b70fe6 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:41.721276 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:47:41.806759 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b683423d-70ae-4c00-9bc7-b07c60b70fe6 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:41.813031 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a901f3b3-fd65-46a1-aca2-b846fa666c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:41.813998 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901f3b3-fd65-46a1-aca2-b846fa666c10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:41.814282 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901f3b3-fd65-46a1-aca2-b846fa666c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:41.821241 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901f3b3-fd65-46a1-aca2-b846fa666c10 admin admin] PATH_INFO: `/v3/users/e9cee90d8dfb418aaca1cee546d93068` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:41.821241 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a901f3b3-fd65-46a1-aca2-b846fa666c10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9cee90d8dfb418aaca1cee546d93068)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:41.821241 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a901f3b3-fd65-46a1-aca2-b846fa666c10 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:41.897120 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a901f3b3-fd65-46a1-aca2-b846fa666c10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9cee90d8dfb418aaca1cee546d93068'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:41.909389 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a901f3b3-fd65-46a1-aca2-b846fa666c10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9cee90d8dfb418aaca1cee546d93068'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:41.910679 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a901f3b3-fd65-46a1-aca2-b846fa666c10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9cee90d8dfb418aaca1cee546d93068'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:41.911703 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a901f3b3-fd65-46a1-aca2-b846fa666c10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9cee90d8dfb418aaca1cee546d93068'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:41.937456 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a901f3b3-fd65-46a1-aca2-b846fa666c10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9cee90d8dfb418aaca1cee546d93068'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:41.959444 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1425/2889] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:47:41 2022] DELETE /identity/v3/users/e9cee90d8dfb418aaca1cee546d93068 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:47:41.964550 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5831e74-ec43-46a2-bcba-1be68635b15c None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:41.965462 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5831e74-ec43-46a2-bcba-1be68635b15c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:41.965462 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5831e74-ec43-46a2-bcba-1be68635b15c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:41.971811 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:47:41.993994 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-f5831e74-ec43-46a2-bcba-1be68635b15c None None] MFA Rules not processed for user `dd94cb8910014827bad3f45b205880f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:47:41.997016 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f5831e74-ec43-46a2-bcba-1be68635b15c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:47:42.090112 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1465/2890] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:47:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:47:42.104224 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a901f3b3-fd65-46a1-aca2-b846fa666c10 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:42.106443 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a901f3b3-fd65-46a1-aca2-b846fa666c10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:47:42.212268 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a901f3b3-fd65-46a1-aca2-b846fa666c10 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:42.218796 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1364a7e-04b7-48f8-8d42-37a9ed854a93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:42.219520 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1364a7e-04b7-48f8-8d42-37a9ed854a93 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:42.219840 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1364a7e-04b7-48f8-8d42-37a9ed854a93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:42.220152 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1364a7e-04b7-48f8-8d42-37a9ed854a93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:42.232211 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1364a7e-04b7-48f8-8d42-37a9ed854a93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:42.233568 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1364a7e-04b7-48f8-8d42-37a9ed854a93 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:42.246016 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1426/2891] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:47:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:47:42.663659 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5831e74-ec43-46a2-bcba-1be68635b15c None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:42.665609 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f5831e74-ec43-46a2-bcba-1be68635b15c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:47:42.685887 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5831e74-ec43-46a2-bcba-1be68635b15c None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:42.694557 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bc2581-364e-4767-8a44-b518aba7ce42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:42.696174 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc2581-364e-4767-8a44-b518aba7ce42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:42.696688 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc2581-364e-4767-8a44-b518aba7ce42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:42.697102 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc2581-364e-4767-8a44-b518aba7ce42 admin admin] PATH_INFO: `/v3/projects/551e24e8fae448ed9674fc7dfc38e880` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:42.699666 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bc2581-364e-4767-8a44-b518aba7ce42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=551e24e8fae448ed9674fc7dfc38e880)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:42.703657 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bc2581-364e-4767-8a44-b518aba7ce42 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:42.788331 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-55bc2581-364e-4767-8a44-b518aba7ce42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '551e24e8fae448ed9674fc7dfc38e880', 'request_id': 'req-55bc2581-364e-4767-8a44-b518aba7ce42'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:42.795365 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-55bc2581-364e-4767-8a44-b518aba7ce42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '551e24e8fae448ed9674fc7dfc38e880', 'request_id': 'req-55bc2581-364e-4767-8a44-b518aba7ce42'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:42.805694 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-55bc2581-364e-4767-8a44-b518aba7ce42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '551e24e8fae448ed9674fc7dfc38e880', 'request_id': 'req-55bc2581-364e-4767-8a44-b518aba7ce42'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:42.819330 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-55bc2581-364e-4767-8a44-b518aba7ce42 admin admin] The token cache is being invalidate because project 551e24e8fae448ed9674fc7dfc38e880 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:47:42.819624 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-55bc2581-364e-4767-8a44-b518aba7ce42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:42.820397 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-55bc2581-364e-4767-8a44-b518aba7ce42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:42.822084 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1466/2892] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:47:42 2022] DELETE /identity/v3/projects/551e24e8fae448ed9674fc7dfc38e880 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:47:52.240328 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=70,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:47:52.816149 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=53,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:48:05.295715 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1364a7e-04b7-48f8-8d42-37a9ed854a93 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:48:05.297513 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a1364a7e-04b7-48f8-8d42-37a9ed854a93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:48:05.384584 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1364a7e-04b7-48f8-8d42-37a9ed854a93 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:48:05.456718 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f06f82-7d0c-4610-ad43-5baf73692363 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef1d49b36cdc4665b82487be97e9dcd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:48:05.457490 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f06f82-7d0c-4610-ad43-5baf73692363 service placement] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:05.457820 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f06f82-7d0c-4610-ad43-5baf73692363 service placement] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:05.458102 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f06f82-7d0c-4610-ad43-5baf73692363 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:05.460236 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-48f06f82-7d0c-4610-ad43-5baf73692363 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:48:05.473406 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f06f82-7d0c-4610-ad43-5baf73692363 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:48:05.475099 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f06f82-7d0c-4610-ad43-5baf73692363 service placement] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:48:05.486559 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1427/2893] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:48:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:48:15.481606 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=58 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:48:24.656995 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bc2581-364e-4767-8a44-b518aba7ce42 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:48:24.658937 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-55bc2581-364e-4767-8a44-b518aba7ce42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:48:24.748218 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bc2581-364e-4767-8a44-b518aba7ce42 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:48:24.752121 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09139c91-c106-4b9e-a377-2db67738fe3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b683e6f78abd4986b415eb9ddf34cb8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:48:24.752806 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09139c91-c106-4b9e-a377-2db67738fe3a service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:24.753085 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09139c91-c106-4b9e-a377-2db67738fe3a service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:24.753534 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09139c91-c106-4b9e-a377-2db67738fe3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:24.756268 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-09139c91-c106-4b9e-a377-2db67738fe3a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:48:24.839490 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09139c91-c106-4b9e-a377-2db67738fe3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:48:24.841469 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09139c91-c106-4b9e-a377-2db67738fe3a service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:48:24.854828 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1467/2894] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:48:24 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:48:26.204530 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f06f82-7d0c-4610-ad43-5baf73692363 service placement] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:48:26.206770 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-48f06f82-7d0c-4610-ad43-5baf73692363 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:48:26.278883 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f06f82-7d0c-4610-ad43-5baf73692363 service placement] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:48:26.283009 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcda703-4ef7-4218-b49b-340abdbc4d5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:48:26.283681 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcda703-4ef7-4218-b49b-340abdbc4d5b service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:26.283946 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcda703-4ef7-4218-b49b-340abdbc4d5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:26.284296 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcda703-4ef7-4218-b49b-340abdbc4d5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:26.286685 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-dfcda703-4ef7-4218-b49b-340abdbc4d5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:48:26.353731 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcda703-4ef7-4218-b49b-340abdbc4d5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:48:26.355145 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcda703-4ef7-4218-b49b-340abdbc4d5b service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:48:26.365495 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1428/2895] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:48:26 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:48:27.849791 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09139c91-c106-4b9e-a377-2db67738fe3a service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:48:27.851767 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-09139c91-c106-4b9e-a377-2db67738fe3a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:48:27.969277 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09139c91-c106-4b9e-a377-2db67738fe3a service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:48:27.977411 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb30d0-c026-4397-a353-1ead47e79617 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1b3fa67905c4027b3656ac36e2b76c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:48:27.978165 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb30d0-c026-4397-a353-1ead47e79617 service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:27.978507 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb30d0-c026-4397-a353-1ead47e79617 service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:27.978819 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb30d0-c026-4397-a353-1ead47e79617 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:27.981108 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7abb30d0-c026-4397-a353-1ead47e79617 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:48:27.999328 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abb30d0-c026-4397-a353-1ead47e79617 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:48:28.001408 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abb30d0-c026-4397-a353-1ead47e79617 service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:48:28.016519 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1468/2896] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:48:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:48:36.315949 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124550ca-f075-4020-8fcb-77a9d151a85d None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:36.316616 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124550ca-f075-4020-8fcb-77a9d151a85d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:36.317066 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124550ca-f075-4020-8fcb-77a9d151a85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:36.350379 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-124550ca-f075-4020-8fcb-77a9d151a85d None None] MFA Rules not processed for user `0628da68245a42e4af2e16e42babc15a`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:48:36.352170 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-124550ca-f075-4020-8fcb-77a9d151a85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:48:36.361875 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1429/2897] 10.222.0.87 () {58 vars in 957 bytes} [Tue Jun 21 18:48:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:48:36.783394 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb30d0-c026-4397-a353-1ead47e79617 service nova] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:48:36.786531 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7abb30d0-c026-4397-a353-1ead47e79617 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:48:36.804380 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb30d0-c026-4397-a353-1ead47e79617 service nova] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:48:36.813435 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d0acf7-a651-4de8-a0c5-2d39bf33e256 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:48:36.813435 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d0acf7-a651-4de8-a0c5-2d39bf33e256 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:36.813435 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d0acf7-a651-4de8-a0c5-2d39bf33e256 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:36.813435 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d0acf7-a651-4de8-a0c5-2d39bf33e256 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:36.814103 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-90d0acf7-a651-4de8-a0c5-2d39bf33e256 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:48:36.827969 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d0acf7-a651-4de8-a0c5-2d39bf33e256 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:48:36.829496 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d0acf7-a651-4de8-a0c5-2d39bf33e256 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:48:36.842531 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1469/2898] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:48:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:48:38.750883 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124550ca-f075-4020-8fcb-77a9d151a85d None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:48:38.764370 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124550ca-f075-4020-8fcb-77a9d151a85d None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:48:38.771162 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55526783-8a0c-482e-81dd-724bcd4848b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:48:38.771162 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55526783-8a0c-482e-81dd-724bcd4848b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:38.771162 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55526783-8a0c-482e-81dd-724bcd4848b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:38.771162 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55526783-8a0c-482e-81dd-724bcd4848b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:38.773848 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-55526783-8a0c-482e-81dd-724bcd4848b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:48:38.877669 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55526783-8a0c-482e-81dd-724bcd4848b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:48:38.879653 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55526783-8a0c-482e-81dd-724bcd4848b1 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:48:38.892817 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1430/2899] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:48:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:48:46.614427 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d0acf7-a651-4de8-a0c5-2d39bf33e256 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:48:46.615971 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-90d0acf7-a651-4de8-a0c5-2d39bf33e256 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:48:46.632597 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d0acf7-a651-4de8-a0c5-2d39bf33e256 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:48:46.637895 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f107c1-9219-4a05-8921-3de69d245363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:48:46.638904 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f107c1-9219-4a05-8921-3de69d245363 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:46.639187 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f107c1-9219-4a05-8921-3de69d245363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:46.639464 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f107c1-9219-4a05-8921-3de69d245363 admin admin] PATH_INFO: `/v3/users/ea95ad1fabb74ba692aa9172128035eb` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:46.640858 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f107c1-9219-4a05-8921-3de69d245363 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea95ad1fabb74ba692aa9172128035eb)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:48:46.643056 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f107c1-9219-4a05-8921-3de69d245363 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:48:46.688966 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:48:46.716933 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-07f107c1-9219-4a05-8921-3de69d245363 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea95ad1fabb74ba692aa9172128035eb'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:48:46.736200 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-07f107c1-9219-4a05-8921-3de69d245363 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea95ad1fabb74ba692aa9172128035eb'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:48:46.737118 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-07f107c1-9219-4a05-8921-3de69d245363 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea95ad1fabb74ba692aa9172128035eb'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:48:46.738435 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-07f107c1-9219-4a05-8921-3de69d245363 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea95ad1fabb74ba692aa9172128035eb'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:48:46.748615 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-07f107c1-9219-4a05-8921-3de69d245363 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea95ad1fabb74ba692aa9172128035eb'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:48:46.772562 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1470/2900] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:48:46 2022] DELETE /identity/v3/users/ea95ad1fabb74ba692aa9172128035eb => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:48:47.020211 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55526783-8a0c-482e-81dd-724bcd4848b1 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:48:47.042839 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55526783-8a0c-482e-81dd-724bcd4848b1 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:48:47.092401 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:48:47.116804 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f40160a-9621-47a6-a02d-ffd2d7353127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:48:47.117739 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40160a-9621-47a6-a02d-ffd2d7353127 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:47.118022 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40160a-9621-47a6-a02d-ffd2d7353127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:47.118296 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40160a-9621-47a6-a02d-ffd2d7353127 admin admin] PATH_INFO: `/v3/projects/990014bc19574d66ae5faae5c0751d5c` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:47.119604 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f40160a-9621-47a6-a02d-ffd2d7353127 admin admin] RBAC: Authorizing `identity:delete_project(project_id=990014bc19574d66ae5faae5c0751d5c)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:48:47.121542 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f40160a-9621-47a6-a02d-ffd2d7353127 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:48:47.180040 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8f40160a-9621-47a6-a02d-ffd2d7353127 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '990014bc19574d66ae5faae5c0751d5c', 'request_id': 'req-8f40160a-9621-47a6-a02d-ffd2d7353127'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:48:47.189694 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8f40160a-9621-47a6-a02d-ffd2d7353127 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '990014bc19574d66ae5faae5c0751d5c', 'request_id': 'req-8f40160a-9621-47a6-a02d-ffd2d7353127'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:48:47.197112 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8f40160a-9621-47a6-a02d-ffd2d7353127 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '990014bc19574d66ae5faae5c0751d5c', 'request_id': 'req-8f40160a-9621-47a6-a02d-ffd2d7353127'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:48:47.211509 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8f40160a-9621-47a6-a02d-ffd2d7353127 admin admin] The token cache is being invalidate because project 990014bc19574d66ae5faae5c0751d5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:48:47.212034 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8f40160a-9621-47a6-a02d-ffd2d7353127 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:48:47.213281 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8f40160a-9621-47a6-a02d-ffd2d7353127 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:48:47.215029 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1431/2901] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:48:47 2022] DELETE /identity/v3/projects/990014bc19574d66ae5faae5c0751d5c => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:48:47.219568 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f107c1-9219-4a05-8921-3de69d245363 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:48:47.221180 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-07f107c1-9219-4a05-8921-3de69d245363 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:48:47.239458 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f107c1-9219-4a05-8921-3de69d245363 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:48:47.309886 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935bd546-d91d-4b8f-a807-62306263e904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:48:47.310794 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935bd546-d91d-4b8f-a807-62306263e904 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:47.311065 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935bd546-d91d-4b8f-a807-62306263e904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:47.311353 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935bd546-d91d-4b8f-a807-62306263e904 admin admin] PATH_INFO: `/v3/users/f1b5e6da52994804b1859c22cb739b06` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:47.312632 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935bd546-d91d-4b8f-a807-62306263e904 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1b5e6da52994804b1859c22cb739b06)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:48:47.314715 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935bd546-d91d-4b8f-a807-62306263e904 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:48:47.390230 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-935bd546-d91d-4b8f-a807-62306263e904 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1b5e6da52994804b1859c22cb739b06'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:48:47.407274 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-935bd546-d91d-4b8f-a807-62306263e904 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1b5e6da52994804b1859c22cb739b06'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:48:47.408026 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-935bd546-d91d-4b8f-a807-62306263e904 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1b5e6da52994804b1859c22cb739b06'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:48:47.408666 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-935bd546-d91d-4b8f-a807-62306263e904 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1b5e6da52994804b1859c22cb739b06'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:48:47.419528 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-935bd546-d91d-4b8f-a807-62306263e904 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1b5e6da52994804b1859c22cb739b06'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:48:47.438026 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1471/2902] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:48:47 2022] DELETE /identity/v3/users/f1b5e6da52994804b1859c22cb739b06 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:48:47.651268 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f40160a-9621-47a6-a02d-ffd2d7353127 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:48:47.652944 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8f40160a-9621-47a6-a02d-ffd2d7353127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:48:47.669728 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f40160a-9621-47a6-a02d-ffd2d7353127 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:48:47.750365 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35af0996-d9f9-4b40-b923-0204cf91bc40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:48:47.751255 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af0996-d9f9-4b40-b923-0204cf91bc40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:47.751538 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af0996-d9f9-4b40-b923-0204cf91bc40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:47.751817 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af0996-d9f9-4b40-b923-0204cf91bc40 admin admin] PATH_INFO: `/v3/projects/51a74b58d7534f4b9a2b613c9c1bf1ca` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:47.753131 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35af0996-d9f9-4b40-b923-0204cf91bc40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51a74b58d7534f4b9a2b613c9c1bf1ca)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:48:47.755540 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35af0996-d9f9-4b40-b923-0204cf91bc40 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:48:47.822678 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-35af0996-d9f9-4b40-b923-0204cf91bc40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51a74b58d7534f4b9a2b613c9c1bf1ca', 'request_id': 'req-35af0996-d9f9-4b40-b923-0204cf91bc40'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:48:47.833003 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-35af0996-d9f9-4b40-b923-0204cf91bc40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51a74b58d7534f4b9a2b613c9c1bf1ca', 'request_id': 'req-35af0996-d9f9-4b40-b923-0204cf91bc40'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:48:47.840102 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-35af0996-d9f9-4b40-b923-0204cf91bc40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51a74b58d7534f4b9a2b613c9c1bf1ca', 'request_id': 'req-35af0996-d9f9-4b40-b923-0204cf91bc40'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:48:47.854538 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-35af0996-d9f9-4b40-b923-0204cf91bc40 admin admin] The token cache is being invalidate because project 51a74b58d7534f4b9a2b613c9c1bf1ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:48:47.854807 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-35af0996-d9f9-4b40-b923-0204cf91bc40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:48:47.855536 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-35af0996-d9f9-4b40-b923-0204cf91bc40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:48:47.857162 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1432/2903] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:48:47 2022] DELETE /identity/v3/projects/51a74b58d7534f4b9a2b613c9c1bf1ca => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:48:52.726458 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935bd546-d91d-4b8f-a807-62306263e904 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:48:52.728614 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-935bd546-d91d-4b8f-a807-62306263e904 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:48:52.753916 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:48:52.802493 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935bd546-d91d-4b8f-a807-62306263e904 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:48:52.867102 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc110ea-f17c-444c-93df-0d780485f9e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:48:52.867675 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc110ea-f17c-444c-93df-0d780485f9e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:52.867970 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc110ea-f17c-444c-93df-0d780485f9e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:52.868248 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc110ea-f17c-444c-93df-0d780485f9e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:52.870409 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7bc110ea-f17c-444c-93df-0d780485f9e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:48:52.960785 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc110ea-f17c-444c-93df-0d780485f9e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:48:52.962424 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc110ea-f17c-444c-93df-0d780485f9e5 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:48:52.975483 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1472/2904] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:48:52 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:48:57.850876 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:02.969246 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=62 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:04.279414 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35af0996-d9f9-4b40-b923-0204cf91bc40 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:04.281832 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-35af0996-d9f9-4b40-b923-0204cf91bc40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:04.364898 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35af0996-d9f9-4b40-b923-0204cf91bc40 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:04.370725 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26052152-9818-4026-8a3f-5b2794a1b839 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:04.371923 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26052152-9818-4026-8a3f-5b2794a1b839 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:04.372448 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26052152-9818-4026-8a3f-5b2794a1b839 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:04.372959 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26052152-9818-4026-8a3f-5b2794a1b839 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:04.376613 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-26052152-9818-4026-8a3f-5b2794a1b839 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:04.395840 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26052152-9818-4026-8a3f-5b2794a1b839 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:04.397352 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26052152-9818-4026-8a3f-5b2794a1b839 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:04.407478 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1433/2905] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:50:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:50:14.403094 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=19 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:59.894351 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc110ea-f17c-444c-93df-0d780485f9e5 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:59.945131 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc110ea-f17c-444c-93df-0d780485f9e5 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:59.949467 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87846581-9e3b-4ca7-86c3-828c628a68da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:59.950414 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87846581-9e3b-4ca7-86c3-828c628a68da service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:59.950561 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87846581-9e3b-4ca7-86c3-828c628a68da service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:59.950718 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87846581-9e3b-4ca7-86c3-828c628a68da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:59.952671 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-87846581-9e3b-4ca7-86c3-828c628a68da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:51:00.108657 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87846581-9e3b-4ca7-86c3-828c628a68da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:00.110212 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87846581-9e3b-4ca7-86c3-828c628a68da service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:00.123048 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1473/2906] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:50:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:51:01.318494 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26052152-9818-4026-8a3f-5b2794a1b839 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:01.347647 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26052152-9818-4026-8a3f-5b2794a1b839 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:01.352234 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e089d2-9172-4e0d-8496-b189c7dab5b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:01.352916 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e089d2-9172-4e0d-8496-b189c7dab5b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:01.353203 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e089d2-9172-4e0d-8496-b189c7dab5b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:01.353491 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e089d2-9172-4e0d-8496-b189c7dab5b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:01.355563 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-77e089d2-9172-4e0d-8496-b189c7dab5b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:51:01.371597 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e089d2-9172-4e0d-8496-b189c7dab5b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:01.373289 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e089d2-9172-4e0d-8496-b189c7dab5b2 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:01.386717 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1434/2907] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:51:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:51:10.117537 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=58 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:11.380663 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=10 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:13.227392 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87846581-9e3b-4ca7-86c3-828c628a68da service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:13.228973 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-87846581-9e3b-4ca7-86c3-828c628a68da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:51:13.265826 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87846581-9e3b-4ca7-86c3-828c628a68da service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:13.270721 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec84b0af-f270-4ee9-8564-adc6569caa32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '480e2529015c4c2fb4d6cd1985009fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:13.271303 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec84b0af-f270-4ee9-8564-adc6569caa32 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:13.271586 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec84b0af-f270-4ee9-8564-adc6569caa32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:13.271857 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec84b0af-f270-4ee9-8564-adc6569caa32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:13.273928 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ec84b0af-f270-4ee9-8564-adc6569caa32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:51:13.373050 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec84b0af-f270-4ee9-8564-adc6569caa32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:13.374571 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec84b0af-f270-4ee9-8564-adc6569caa32 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:13.397268 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1474/2908] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:51:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:51:20.843723 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e089d2-9172-4e0d-8496-b189c7dab5b2 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:20.845294 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-77e089d2-9172-4e0d-8496-b189c7dab5b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:51:20.911081 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec84b0af-f270-4ee9-8564-adc6569caa32 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:20.913115 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ec84b0af-f270-4ee9-8564-adc6569caa32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:51:20.929961 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec84b0af-f270-4ee9-8564-adc6569caa32 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:20.937815 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0968dd-2602-424f-b170-e6b656b82ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:20.939218 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0968dd-2602-424f-b170-e6b656b82ac3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:20.939688 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0968dd-2602-424f-b170-e6b656b82ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:20.940156 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0968dd-2602-424f-b170-e6b656b82ac3 admin admin] PATH_INFO: `/v3/users/0628da68245a42e4af2e16e42babc15a` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:20.942208 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0968dd-2602-424f-b170-e6b656b82ac3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0628da68245a42e4af2e16e42babc15a)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:20.945766 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0968dd-2602-424f-b170-e6b656b82ac3 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:20.946955 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e089d2-9172-4e0d-8496-b189c7dab5b2 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:20.952268 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ebcc6-b8b4-42d5-a4fc-194147e75f92 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef1d49b36cdc4665b82487be97e9dcd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:20.953789 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ebcc6-b8b4-42d5-a4fc-194147e75f92 service placement] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:20.954120 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ebcc6-b8b4-42d5-a4fc-194147e75f92 service placement] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:20.954384 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ebcc6-b8b4-42d5-a4fc-194147e75f92 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:20.956334 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-502ebcc6-b8b4-42d5-a4fc-194147e75f92 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:51:20.969349 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502ebcc6-b8b4-42d5-a4fc-194147e75f92 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:20.971177 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502ebcc6-b8b4-42d5-a4fc-194147e75f92 service placement] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:20.990445 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1435/2909] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:51:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:51:21.019578 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0c0968dd-2602-424f-b170-e6b656b82ac3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0628da68245a42e4af2e16e42babc15a'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:51:21.037182 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0c0968dd-2602-424f-b170-e6b656b82ac3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0628da68245a42e4af2e16e42babc15a'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:51:21.037988 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0c0968dd-2602-424f-b170-e6b656b82ac3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0628da68245a42e4af2e16e42babc15a'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:51:21.039299 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0c0968dd-2602-424f-b170-e6b656b82ac3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0628da68245a42e4af2e16e42babc15a'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:51:21.048454 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0c0968dd-2602-424f-b170-e6b656b82ac3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0628da68245a42e4af2e16e42babc15a'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:51:21.063619 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1475/2910] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:51:20 2022] DELETE /identity/v3/users/0628da68245a42e4af2e16e42babc15a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:51:21.262581 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ebcc6-b8b4-42d5-a4fc-194147e75f92 service placement] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:21.278255 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ebcc6-b8b4-42d5-a4fc-194147e75f92 service placement] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:21.346533 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7773805d-ffa0-48d5-a244-c36878e98f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:21.347320 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773805d-ffa0-48d5-a244-c36878e98f89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:21.347615 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773805d-ffa0-48d5-a244-c36878e98f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:21.347908 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773805d-ffa0-48d5-a244-c36878e98f89 admin admin] PATH_INFO: `/v3/projects/88c0ee0e3bda4c9b98c295af8718eff6` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:21.349150 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7773805d-ffa0-48d5-a244-c36878e98f89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88c0ee0e3bda4c9b98c295af8718eff6)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:21.351149 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7773805d-ffa0-48d5-a244-c36878e98f89 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:21.409860 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7773805d-ffa0-48d5-a244-c36878e98f89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88c0ee0e3bda4c9b98c295af8718eff6', 'request_id': 'req-7773805d-ffa0-48d5-a244-c36878e98f89'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:51:21.419997 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7773805d-ffa0-48d5-a244-c36878e98f89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88c0ee0e3bda4c9b98c295af8718eff6', 'request_id': 'req-7773805d-ffa0-48d5-a244-c36878e98f89'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:51:21.428346 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7773805d-ffa0-48d5-a244-c36878e98f89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88c0ee0e3bda4c9b98c295af8718eff6', 'request_id': 'req-7773805d-ffa0-48d5-a244-c36878e98f89'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:51:21.441723 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7773805d-ffa0-48d5-a244-c36878e98f89 admin admin] The token cache is being invalidate because project 88c0ee0e3bda4c9b98c295af8718eff6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:51:21.442084 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7773805d-ffa0-48d5-a244-c36878e98f89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:51:21.443041 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7773805d-ffa0-48d5-a244-c36878e98f89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:51:21.445029 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1436/2911] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:51:21 2022] DELETE /identity/v3/projects/88c0ee0e3bda4c9b98c295af8718eff6 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:51:21.449691 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0968dd-2602-424f-b170-e6b656b82ac3 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:21.451267 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-0c0968dd-2602-424f-b170-e6b656b82ac3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:51:21.465363 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0968dd-2602-424f-b170-e6b656b82ac3 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:21.495534 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:21.551667 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba9c398-b6e6-440d-bb34-f36dd0a15997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:21.552578 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba9c398-b6e6-440d-bb34-f36dd0a15997 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:21.552868 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba9c398-b6e6-440d-bb34-f36dd0a15997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:21.553228 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba9c398-b6e6-440d-bb34-f36dd0a15997 admin admin] PATH_INFO: `/v3/users/f620a64d26a541d788defef196a0eb91` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:21.554547 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba9c398-b6e6-440d-bb34-f36dd0a15997 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f620a64d26a541d788defef196a0eb91)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:21.556747 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba9c398-b6e6-440d-bb34-f36dd0a15997 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:21.615342 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3ba9c398-b6e6-440d-bb34-f36dd0a15997 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f620a64d26a541d788defef196a0eb91'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:51:21.632359 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3ba9c398-b6e6-440d-bb34-f36dd0a15997 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f620a64d26a541d788defef196a0eb91'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:51:21.633261 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3ba9c398-b6e6-440d-bb34-f36dd0a15997 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f620a64d26a541d788defef196a0eb91'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:51:21.633974 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3ba9c398-b6e6-440d-bb34-f36dd0a15997 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f620a64d26a541d788defef196a0eb91'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:51:21.643189 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3ba9c398-b6e6-440d-bb34-f36dd0a15997 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f620a64d26a541d788defef196a0eb91'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:51:21.657179 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1476/2912] 10.222.0.87 () {58 vars in 1223 bytes} [Tue Jun 21 18:51:21 2022] DELETE /identity/v3/users/f620a64d26a541d788defef196a0eb91 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:51:22.135736 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7773805d-ffa0-48d5-a244-c36878e98f89 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:22.137234 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7773805d-ffa0-48d5-a244-c36878e98f89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:51:22.151999 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7773805d-ffa0-48d5-a244-c36878e98f89 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:22.175567 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:22.224920 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af29146e-6fd9-488f-a193-b10336f1314d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd94cb8910014827bad3f45b205880f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58da051d3b4a47fab4c706fb517e9ae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:22.225870 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29146e-6fd9-488f-a193-b10336f1314d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:22.226159 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29146e-6fd9-488f-a193-b10336f1314d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:22.227800 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29146e-6fd9-488f-a193-b10336f1314d admin admin] PATH_INFO: `/v3/projects/39e16f89051b42489c6d7775bef204a4` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:22.229122 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af29146e-6fd9-488f-a193-b10336f1314d admin admin] RBAC: Authorizing `identity:delete_project(project_id=39e16f89051b42489c6d7775bef204a4)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:22.231192 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af29146e-6fd9-488f-a193-b10336f1314d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:22.291383 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-af29146e-6fd9-488f-a193-b10336f1314d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39e16f89051b42489c6d7775bef204a4', 'request_id': 'req-af29146e-6fd9-488f-a193-b10336f1314d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:51:22.299842 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-af29146e-6fd9-488f-a193-b10336f1314d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39e16f89051b42489c6d7775bef204a4', 'request_id': 'req-af29146e-6fd9-488f-a193-b10336f1314d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:51:22.306408 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-af29146e-6fd9-488f-a193-b10336f1314d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39e16f89051b42489c6d7775bef204a4', 'request_id': 'req-af29146e-6fd9-488f-a193-b10336f1314d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:51:22.319543 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-af29146e-6fd9-488f-a193-b10336f1314d admin admin] The token cache is being invalidate because project 39e16f89051b42489c6d7775bef204a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:51:22.319852 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-af29146e-6fd9-488f-a193-b10336f1314d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:51:22.320572 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-af29146e-6fd9-488f-a193-b10336f1314d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:51:22.322372 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1437/2913] 10.222.0.87 () {58 vars in 1232 bytes} [Tue Jun 21 18:51:22 2022] DELETE /identity/v3/projects/39e16f89051b42489c6d7775bef204a4 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:51:31.652958 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=41,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:32.319651 dv-cinder-844601-4-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:09.540269 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba9c398-b6e6-440d-bb34-f36dd0a15997 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:09.541812 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3ba9c398-b6e6-440d-bb34-f36dd0a15997 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:53:09.649109 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba9c398-b6e6-440d-bb34-f36dd0a15997 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:09.718997 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb9d59a-dd2f-4dbc-a450-d0b52021b368 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef1d49b36cdc4665b82487be97e9dcd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f30c3ddb5e44e9299ff228fe94c27fb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:09.719679 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9d59a-dd2f-4dbc-a450-d0b52021b368 service placement] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:09.719963 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9d59a-dd2f-4dbc-a450-d0b52021b368 service placement] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:09.720249 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9d59a-dd2f-4dbc-a450-d0b52021b368 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:09.722612 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5cb9d59a-dd2f-4dbc-a450-d0b52021b368 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:53:09.739502 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb9d59a-dd2f-4dbc-a450-d0b52021b368 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:09.741280 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb9d59a-dd2f-4dbc-a450-d0b52021b368 service placement] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:09.754581 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1477/2914] 10.222.0.87 () {60 vars in 1365 bytes} [Tue Jun 21 18:53:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:53:19.748052 dv-cinder-844601-4-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=67 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}